jeff eaton

This year, a bumper crop of cucumbers was turning our garden into a bit of a challenge. Catherine and I like cucumbers, see, but the two of us can only consume so many of the things. Last week, there were three of the things sitting on the counter threatening to spoil if we didn't do something fast, and I took matters into my own hands. Some googlng for pickle recipes yielded a few promising hits, and I made up substitutions for stuff we didn't have. ("No fresh garlic in the house? I'll use a few peppercorns...")Three jarfulls of the little suckers have been aging for almost a week, and I finally popped one open to see how they turned out. They're certainly not up to Vlasic standards -- not enough spice, a little too much sugar and vinegar -- but all things considered, they're not too bad. Hooray for pickles! I feel terribly domestic, now.

chx and I spent some time this evening talking over the tricky connundrum of multipart (multi-step, multi-page, whatever you want to call them) forms in Drupal. Generally, they tend to fall into one of the following three categories:

1. A really huge form divided up into multiple steps. In this scenerio you probably want to validate at each step, but need to hold onto the values (perhaps in hidden fields) until all the steps are completed and the data can be processed.
2. A chain of forms that each complete and move on to the next. Each one should validate, submit, and hand off control to the next step, which may differ depending on the values submitted in the previous step.
3. Forms that build themselves. You do enter values, submit, and more fields (or different) appear in the form based on what you submitted. You submit again, and again, and so on until it's "done" and then the real work of processing the form is done.

The problem is that drupal_get_form(), that workhorse of form building, validation, and processing, doesn't really lend itself to forms that are in any way dynamic. To understand why (and understand what may be a solution), we'll first revisit the current form workflow.

1. Your function, mymodule_page(), gets called. It constructs a form array and passes it to drupal_get_form().
2. drupal_get_form() looks to see if there is an incoming set of form values in the $_POST variable. At this point, the answer is no.
3. drupal_get_form() skips over any validation or submission logic and just renders the $form array into HTML and spits it out.
4. the user fills in values and clicks SUBMIT. By default, It's pointing to the same page, so your mymodule_page() function gets called again!
5. Your mymodule_page() function builds its $form array again, and calls drupal_get_form() again.
6. This time, though, there IS an incoming set of values in the $_POST variable. drupal_get_form() (via a helper function) takes the $form array that your mymodule_page() function passed in, and walks through it, putting the incoming values into the proper slots in the $form array.
7. drupal_get_form() passes things on to drupal_validate_form(), which uses the passed-in $form, and the incoming $form_values, to make sure everything is kosher.
8. If things aren't kosher, it just just spits out the rendered version of the form with the errors highlighted.
9. If things ARE kosher, it passes things on to drupal_submit_form(), which handles actually processing the form, saving data, etc. If a 'destination' page has been specified for the form, it redirects the browser to it. If not, it renders the form.

That works smashingly well for single page forms where you enter all of your data, submit, correct any errors, then submit again. But, as we said, the hangup is in step number 6, there. See it? It's sneaky. It's using a set of submitted values from the FIRST time you built the form in step 1, but validating them against the SECOND copy of the form that you built in step 5. This works fine if it's always the same, but in cases where the form is dynamic, there will be a mismatch between the two and it will always fail validation. There are ways around it that work in certain situations, but they all tend to fall down when faced with the demanding workflow of full-sized dynamic and multipart forms.

It appears, though, that dopry's post on groups.drupal.org may be the key to the solution. What is it? That's coming soon...

To those who aren't drupal geeks, I apologize. The last run of blog posts has been... well... Terribly Drupally. And unfortunately, that's not going to change for a bit. There's lots of fun stuff to blog about, and Catherine and I are having adventures, but right now it's getting close to Drupal's autumn code freeze and it's time to get geekier. So without further ado, the solution to the node rendering connundrum presented in my last post.

In it, I outlined the problems with the current node_view() system and nodeapi's 'view' operation. Like the old pre-Drupal-4.7 form building functions, it slaps together HTML chunks and spits out a big one. Dealing with things on a more granular level just isn't possible. This makes things a hassle for themers, and completely breaks certain modules (like the popular inline) when themers try to work around it.

Fortunately, the solution that we built for form rendering in version 4.7 is also the solution for node content. It might not be obvious, but our form rendering infrastructure is really designed to operate on any kind of content stored in nested structured arrays. The bits of data can describe file attachments and recipe ingredients just as easily as radio buttons and dropdown boxes, our core rendering function just eats the arrays up, calls the specified theming functions, and spits them out. Adrian and the other FormAPI gurus designed it with generalized content-rendering in mind, but weren't able to get those features into core in time for 4.7.

Well, now it's time: there's a patch available for review that changes the node module to use intelligently structured arrays for its content rendering.

How does it work? Moshe from #drupal asked me to blog a bit about it, so... here goes.

UPDATE: RobRoy from #drupal has been chewing on some of the same problems outlined in the post below. There's an issue on drupal.org for anyone interested in helping tackle the building-nodes-cleanly problem.

It seems like the dust has only just settled on Drupal 4.7, and already things are gearing up for major improvements in version 4.8/5.0. In addition to some subtle but important improvements to FormAPI, a new installer, a much improved administration interface, and assorted other goodies, there's a great patch will give site admins the ability to define their own content types. They'll do this right from thae admin page, without writing a single line of code or hacking any existing modules.

Custom content types have been around for ages in the form of FlexiNode, and the Content Creation Kit project has been in gestation for (at least?) a year now. Both FlexiNode and CCK are designed for very complex cases, though. A content type can have text fields, images, file upload fields, multi-select number fields, checkboxes, and all sorts of niftiness. CCK promises major flexibility and performance advanatages over the relatively brittle FlexiNode, but it's still quite a ways from the finish line. In particular, it doesn't yet support the same variety of custom fields that FlexiNode does, due to its relative youth. It's made a lot of progress, but isn't ready for prime time on most sites.

For many sites, though, that's not a problem. They don't need to create wild and crazy content types with five URLs, two pictures, five select boxes and three text fields. They just need to make an 'article' content type, or an 'interview' content type, that's separate from the 'story' and 'blog' types Drupal ships with. They want to make simple content types, and both CCK and Flexinode are like swatting a fly with a Buick.

That's where the patch called 'Prepare the way for CCK' comes in. Rather than focusing on arbitrarily complex content types with custom fields, esoteric data, and so on, it just allows admins to define 'simple' content types with a title, a teaser, and a body. Most content is like that -- blog entries, stories, book pages, and so on all use that simple approach. In addition, those fields are the ones that Drupal automatically maintains for every node in the system, so everything already knows how to work with the new content types. It's a Good Thimg.

In fact, this new feature will do a good chunk of the work that FlexiNode and CCK have had to carry themselves. In fact, it originally descends from the code that CCK uses to handle its basic content-type management. Although FlexiNode's codebase is probably too old to benefit from this, CCK is still 'wet paint' so to speak, and is in a good position to take advantage of it. Once it goes into core, all CCK needs to worry about is adding its own special and spiffy fields to custom content types -- core code will handle managing the types themselves.

This is very cool. Very, very cool in fact. The only problem, though, is that both CCK and Flexinode expose a fundamental problem in the way that Drupal deals with content. In the database, every node has a 'title', a 'teaser', and a 'body'. It makes sense and everything's happy. When other modules add custom data onto a node, they store it in their own database fields and use the 'view' or 'load' hooks to add it onto $node->body and $node->teaser as needed. One example is the 'Upload' module, which tacks its list of file attachments onto $node->body during the 'view' operation. Unfortunately, this means that the body and teaser fields have to do double-duty. Sometimes, they're user-entered data. Other times, they're a mix of user-entered and module-added data.

Some modules, like CCK and Flexinode in particular, hide the 'body' field from the user entirely. They just let administrators choose which custom fields will be glomped together into the teaser, which will be glomped into the body, and assemble $node->body programmatically. This usually works OK, until you start trying to override the theming for a CCK node by rebuilding $body yourself using the assorted component fields.

Because all the other modules have already done their work, dutifully modifying the $node->body field and getting their additions 'just so,' attempting to change it in your theme will wipe out all the additions they tack on. Using inline.module to put images into the body? They vanish. Using upload.module to attach files? They vanish. It's possible to recreate this stuff in your theming layer by manually triggering each add-on module's hooks, prompting it to do its work a second time, but each time you enable a new module you'll have to add code to handle it in your theme.

This issue has recently bitten a number of people using the excellent Contemplate module. It allows users to use php snippets to control the rendering of CCK nodes. It's terribly useful, but it runs smack into this issue and results in confusion for everyone who encounters it.

The long-term solution to this is straightforward but involves a lot of work for Drupal modules: instead of shoving their content into $node->body as HTML, they should add their additions onto the $node object itself, and allow the node theming code to assemble the final results. That's how CCK manages each of its individual fields, and it works pretty well as long as no other modules are involved. It keeps the original data intact, but provides a 'default'

Moving to that model would require a non-trivial (but not horrible) rewrite of almost every content-oriented module. It was brought up a few months ago on the Drupal devel mailing list, but at the time 4.7 was in labor and no one had time to dedicate to pie-in-the-sky thinking. I'm beginning to think it might be one of the critical parts of the system to look at before the September code freeze that's coming up.

• Visit General Protection Fault

GPF is an old-timer in the webcomics world, but sad to say it jumped the shark a while ago. Websnark details its descent into "Huh?" pretty well. There's still some funny in it, though, between the inexplicable story arcs.