Page Rendering in Drupal 7 presentation, Moshe Weitzman, transcript

Page Rendering in Drupal 7, presentation by Moshe Weitzman at DrupalCon Paris 2009. Transcript.

Changes you can make with this rendering system that was hard with D6 but trivial in D7.

D7 website filled with dummy data -- a couple of modifications. Powered by Drupal on left hand side and on the footer. A capability to look at the contents of the whole page in one array and modify bits and pieces of it as you wish. Taken a powered by Drupal block, for any given page, can just duplicate it over to the left hand sidebar. notion that a block can only one appear that has vexed a lot of us solved by a new hook called hook page alter. Much more flexible than a block being in two regions at the same time -- on every single page so put it wherever you want. Crazy client needs fulfilled by moving pieces of the page around without too much trouble.

Particular node -- Node 53 -- new block called Article tools, with only one link -- add new comment. With just a couple lines of code, takes the node links (attached to the node) and moved them to left hand sidebar. This would be very tricky with previous versions of Drupal -- you have pieces that are attached to the node and they don't like to be attached to the page. They want to be really near the node, so you end up having regions that are secretly in the page and so forth. Here, with hook page alter, you're welcome to just put parts of the page where you want them.

Another example of what you can do with hook page alter -- I went ahead and added an advertisement after the first comment. That kind of thing is pretty tricky in prior versions of Drupal, especially if you want to do it for just some nodes and not the others. Let's say you are a newspaper website and you sell a sponsorship for the sports section of your newspaper, so the sponsor has to appear after the first comment. That is a fairly vexing requirement for a developer in Drupal 6 but not anymore. So, site specific customizations are quite a lot easier for Drupal 7 with hook page alter.

How pages get built in Drupal 7 -- bootstrap (changes not relevant for page rendering), menu execute active handler (job of that to figure out which handler will handle the current url), the page callback function (totally changed for Drupal 7), delivery callback serving up pages on alternate, non HTML formats (Jason, Ajax, Adam), hook page alter, rendering and theming it.

Menu system -- menu execute active handler, which asks the menu system to go and do whatever calculations it needs to, and figure out what is the handler for the current page. the primary calculation same as Drupal 6 - menu router table, path with the highest fit, once the menu system replies about which router item is effective for the current page, there's a new hook in Drupal 7. Hook menu active handler is a chance for all the loaded modules to change everything about the menu router item that was selected for the current page. So, I was thinking of examples of how you might use that -- you might implement this hook and say if the current user has an IP in a certain range, change the access callback to false -- if you don't want to allow someone to see your site, that's one way of blocking them (the Drupal way of blocking them; there are several ways to block an IP address). Or the converse, if someone's on your internet, change the access callback to true. So, maybe the developer wants to do that for his own IP, like I am not subject to menu restrictions, I can see any page on the site. In addition to changing access information, you can change what router page callback is. So if you need dynamic control over what router item is selected you can do that. In Drupal 6, we struggle with Page Manager in the panels, where no such hooks exist. This would be a way to go around this. Another module is OG panels, every manager create custom pages for his group -- very hard to implement in Drupal 6 but with this hook, easy for Drupal 7.
The meat of this talk is really about page callback and how it is supposed to return information. Standard Drupal homepage -- node page default -- this function does a select statement, finds the 10 most recent nodes that were promoted to the home page. In prior Drupal versions, after it has done this, returns a strain to the index on pHp. Works differently now -- now the callbacks returned in array -- Drupal render array. A close cousin of form API arrays, just without the form specific stuff.

Left hand part of graphic 11.37 (min/seconds) in the video -- it is what the page callback is returning on the home page, returning an array, the key is called nodes, contents of nodes are 10 other arrays, just below that another array called pager that goes into the next page of promoted page of arrays. Diving into: Key 116, node 116 on the site. This is what the inside of that array looks like -- 12.32 in the video -- the first item in this array is another array called body. This would have been a string in prior versions of Drupal but body is a field in Drupal 7, and so that's why it's a big string. The next one is called field_image -- also an array that's also an API field. In addition, there are properties of 116 that start with pound (extra field, attached, etc.) The way to think of a pound item in render array is that those are not returning HTML or that they're not ending as HTML; those are properties about what you're building but not the actual contents. So, pound pre-array really means call this function right before you're rendering to HTML, and so here the field API has said please call into this extra weights function. What it does there is that it reorders the elements in the array according to what the administrator has decided on his managed display fields and field API. If we just look down a little bit, we see a links array, and those are the nodelinks 1-116, and there's two sets of them, one that comes from node module and the other that comes from the comments module (in this case). And you might see something that looks familiar, pound theme, the value of that is node, and what that really means is that this 116 array is, much later in the requests, going to end up being rendered by node.tpl.php. If we look at the right hand array, we have something called field_image, and this array is a single field, as in field API, formerly known as CCK, it has a pound theme of field, which finally ends up in field.tpl.php being the renderer for each field, and lots of pound properties. Field API has lots of metadata about each field in these render arrays. To a certain extent, at least in the beginning, you can ignore all the pound properties in the render array and focus on the ones that don't have pound, in this case, there's a 0, and that has been expanded. That is the first value in this field, so the first image that's been affiliated with this 116 node. So page callbacks are now expected to build up these arrays, similar to form API. It may look familiar to Drupal 6 developers here because this is how nodes have been built for at least Drupal 6. Nodes have been using render arrays for a while now, so the concept of render arrays and nodes now goes all the way out to the page.

16.12 -- Let's look at some codes that would have built those arrays that we just looked at. Here I have the function node_load_multiple, that gets past an array of Nids. So the part you're not seeing here is the select statement where we went and got the most recent 10 promoted dents (?) We assume that happened successfully. We load them up and then we call node_view_multiple. And so the real meat here is the nod_view_multiple and the node view that come up later. So, what node_view_multiple does that it iterates over each of those multiples here and it calls node view. And so let's look at what node view is. The purpose of node view is to take a loaded node and build a render array from it. And the way it does that is you can see node --> content, that item gets the value of field attach view. We call the field API ad say, field API you know all of the fields that are attached to this content type, go ahead and put all of your render arrays on node content. That's a whole bunch of code comes back with a big array, just like what we saw for that field_image. Node view also adds the links to the node content array. That's what the next two pieces are (18.07) If we're in teaser mode we add the read more link and we add all of the node links. And then the last piece is what have always called node API view. That is hook node view here. These page callbacks are returning very rich arrays. We have not yet themed the node at all -- this is just meta info we're going to later build links with it, and we're later going to render nodes with them but we have not yet done that. This is one of the supreme benefits of Drupal 7, is that we defer all of the theming as late as possible, so that nice hooks such as hook page alter can make changes on very structured data and if you want to remove a block, or remove node links in certain cases, you can do that and you haven't paid the price of theming that content yet, because we defer all that to as late as possible. So its a performance improvement as well.

19.33 -- summary about page callbacks --
they return render arrays, they no longer return string
(if you return a string it will still work and let some of your old code still work for a while, probably for Drupal 7 and maybe not for Drupal 8 -- the recommended way to return it is with a render array.)

20.17 Returning a render array if you have a pound markup -- that's what the second bullet is about, try not to do that if you have a complicated bit of html that you're returning -- break it up into pieces like that default home page breaks every node into its own render array and puts the pager separately. So, if you have complex things you're rendering please put them into different render arrays and you can adjust them for their pages as needed.
Do not call theme from your page callback to the extent possible, if you do it still works, but what you have done is make life harder for people working on the hook page alter and even people who are working on the template layer -- even they can't do tricks they're expecting to do in Drupal 7.
Two examples -- you call #theme= 'table' now and you don't call theme('table')
use #theme = 'item list', not theme('item list') -- there's a subtle difference there that has to do with deferring the work until later.

22.49
Next on the menu execute active handler, we figure out the delivery callback for the current page. This is a new concept in Drupal 7. The page callback for all router items for all pages is drupal deliver page html by default and only router items that want a different one, there's two built in delivery callbacks: there's the html one and the Ajax one. There's a lot of Ajax in Drupal 7 and it has its own way of rendering through delivery callback and so that's why we introduced this. So delivery callback is mainly about -- okay, we've built up a render array, we haven't themed it at all, before we can theme it we need to know what is your desired output format for this page: do you want this current node on HTML? do you want it on Atom? And Docbook for bookpages and so forth…we added a point here, where we can do some pretty interesting rest services kind of work at the delivery callback there. And this is a new idea in Drupal 7, it's not really fleshed out -- we'll do some work on contrib and probably it'll be a greater thing in Drupal 8.

23.18
I want to stop for a second and talk a little about vocabulary in the page rendering system because people seem to get a little twisted around sometimes. And so, this is the definition of the region in Drupal 7. Regions are promises that are made by a theme, that if you put something in this region, I will print it. The most common thing to put in a region is a block.

24.00
There's something we've had for a very long time, HOOK_block and this is the way that modules make chunks of content available to the CMS.

Not to be confused with -- 24.05, the block.module, an optional UI for putting blocks into regions. The optional part is new to Drupal 7 -- it has always been a required module, and it is no longer one anymore, which is pretty neat…you almost always are going to want some way to put blocks into regions, I think, but other ways are panels and content spaces and so forth. You could say that instead of improving block.module, we made it optional, so that better ideas can percolate.

25.13
There's a new, improved variable called $page -- the page array, and the page callback has been building up the principal part of the page array, and we'll look at the page array a lot right now.

25.34
And a new hook, hook page alter where you make alterations to that array.

25.42
Okay, so the next step in page building in Drupal 7, after delivery callback, is drupal render page. It is a new function and principally does the three things that we are seeing here:
it calls hook page build and then passes that the page array, and then it calls hook page alter and it passes that the page array and then it finally calls drupal render.
Hook page build is a chance for modules to add stuff to the page, this is where block module populates the page array that has all the blocks assigned to each region, and once that hook is done, we have a pretty complete page -- it's not been themed at all, but all of the regions are now populated. And then hook page alter is the new very nice hook that is not so much for contrib authors in my opinion, its really for site builders to make site specific customizations that would otherwise be difficult in parts of Drupal. If you have a customization where you always have to change nodes in a certain way you can still use hook node view to do that, and I would recommend doing that, but when you're starting to do things that are more at the page level like move those node links out to the left hand side-bar, hook page alter is really your friend there. And if you only need to do that on sports nodes than it's a piece of cake, cause it happens on every page view and you can just move things. We'll look at how that code looks in just a second.

27.30 So here's two arrays that we might work with -- at the hook page alter -- and so I want to show what this page array is all about.

Left hand array at the top level -- side bar first, content, footer, page top -- those are regions -- so the page arrays has all of the themes that your page has declared. The contents of those arrays are blocks in most cases.
This right hand graphic is the search form block. Blocks are expected to return arrays as well now. Just like page callbacks, if you return a string from a block, it still works -- this is what the $ page variable looks like. The most important region is called content -- that's required. That is where the block that holds the main page content lives. Unlike prior versions of Drupal, the main content of your page is now a block as well.

Here is the code that I used to make the alterations we saw at the very beginning of this presentation -- 30.01 -- it took me one line to copy that powered by block into another region. The line is pretty interesting, I think -- on the right hand side of this line, there's the page array, and then, digging deeper, into the footer region and the system powered by block -- take that and set it to sidebar first system powered by, so we just created a new block out of thin air in our sidebar first region. When the Drupal render goes to render that, its going to duplicate the contents of the block there. It took me four lines to move the node links from the node itself to the sidebar first region. And here's the code that did that -- 31.05 -- it's a fairly lengthy array that you have to dive into -- this one on the right hand side, page content region system main block and then the array of nodes, the 281 node and this has actually changed since I took the screenshot but the concept is the same. Here we went into a node and took out the links and set that to the links variable. I think I have the current code here -- 31.42 -- Move nodelinks to the sidebar -- so, we took this whole thing, set it to a variable links and then I just put a title on it, then again I went to the sidebar first region, added essentially a new block -- it's something like a block but it's a render array and its value is the links. Since I didn't want it to appear in both places I just set the original one. So that's how you move parts of the node into parts of the page. And here's the last example -- 32.40 -- you have to again go into one of these large page arrays, and find where the thread of comments is and I got a pointer to that and called it comments and then added a new render array to that list of comments and I called the key ad and its a pound markup array. And it has a particular weight -- 0.5 in this case, which is what makes it appear between the first comment and the second comment. The first comment has a weight of 0, and the second comment has a weight of 1. And I had to set pound sorted to default in order to convince Drupal to re-set all the nodes before it renders it. That's how you splice something in, is that you just give it the right weight. So this is the kind of code that you'll be writing in hook page alter. Just build a custom module and you're on your way.

34.08 Another piece. These alterations that I made are only effective if your node was created in the last week. That's what 86400 seconds times 7 is. It's the number of seconds in a week. And that's just a whimsical way of saying you could make any changes you want right at one time. This is not like hook menu alter where you get only one shot at altering things and then it gets saved to the database. This is on every page, do what you want and it's a lot more flexible.

35.10
The next step is a call to drupal_render($page) and that's a deceptively simple function call right there but that call alone takes that massive page array and turns it into a HTML string that is suitable for printing out to the page. That's a recursive function. It recurses down through the page array, and it starts at the top and asks do you have any children that need rendering? And if yes, it goes down the children, down, down, till it is down at node 116 and its rendering its node links. That's the first thing it does because that's all the way down the tree. And when I mean rendering the node links, it really means theming them. It calls the pound theme that was specified for that little node links array and grabs that little bit of html that came out of that theme call, and stashes it in a variable, lets call it content. And then it goes up to its parent and it says, do you need to be themed? And it will theme that and it will pen that to what was just printed out and so forth so it just chews through that whole page array appending strings. Those strings are built by calling theme over and over and over again.
Next, the page gets themed. 37.39

I've focused so far on the flexibility that the new page array offers. The advantages are not just that you can do things without pain, there's performance benefits here, the Drupal render function one of the first things it does it checks to see if it has a cached version of the render array that you're about to render and if it does, it just pulls it out of cache, and keeps going with its business -- so you can cache now at any layer of the rendering pipeline and the prior spots that we have done caching in Drupal 6, namely the page cache and the block cache, are extremely coarse compared to this one. This one works for authenticated users. You can cache the list of reset form topics and that in and of itself might not be that special, but what happens in Drupal 6 if you have a node access control module, the block cache cannot work and if you're logged in the page cache cannot work. You're building that block of new form topics every time and it clearly is kind of nasty to get that. And the reason why the block cache won't work is that potentially every person has a list of nodes that they're allowed to see but in Drupal 7 we use the flexible render cacheing so the cache key in Drupal 7 is the query that we're going to use to go fetch the most recent forum topics. And the query actually include the organic groups that you belong to or the taxonomy terms you belong to, so we can cache with exactly the criteria that determine your access. The first one in the group has to pay the price of building forum topics but everyone else just pulls it out of cache. And so you can come up with your own rules of what your cache keys are and what your expiration times are and what your cache buckets are and so this is a powerfully new spot to do caching in Drupal. It's available through the whole page array, so if you have a complicated block, you just put pound cache on it, or you can cache the whole page if you want by putting a pound cache on the page rendering and unlike the standard page caching, its not just the binary thing of if you're anonymous you get it, and everyone else, you're screwed. You can have lots of groups of everyone else.

(Questions not included in the audio, Moshe answers them)

42.51
Other interesting things about the Drupal render function: we talked about #cache a lot,
there's a new thing called #attached and the point of it is that if you need a style sheet or javascript in order to render the stuff that's in your render array, you no longer call js, css, you just attach the same values that you used to to #attached, and the benefit there is that we now cache that too, so when we have a cache hit, we don't have to calculate the most recent forum nodes, we can still put the right CSS on the page to render. If you were calling js or css while querying the forum nodes, your js and css didn't get added, and it would look terrible because the css wasn't there. As long as you use the new #attached, you are compliant with the cacheing of Drupal render. And #theme has had a bunch of changes in Drupal 7 and there's a new theme called theme_wrappers which is interesting.

Changes in the theme layer -- 44.26 -- this slide takes code from no.tpl.php, the design of nodes in garlands is that garland is the default theme and wants to show comments below the node body and node links below the node boy and so what we're doing here is calling a new function called hide and hiding part of the content array, and we hide the comments and hide the links because we're not ready to show those, and we print and render content -- so everything that was is content is going to get printed there, except for stuff that was explicity hidden and then we call render links and render the comments. So for a long time you've been able to print little pieces of the node where you want to, but once you started doing that, you hard coded your node presentation and if someone installed five star and lots of other modules later you were no longer printing the $content variable so those wouldn't show up and you've essentially broken your site because someone's trying to enable a module and it's not working because of your theme. And we solve that pretty cleanly with this setup here in drupal 7 -- render the content array when you want to, and if you want to do stuff afterwards you just hide it, render the content array and then show it with the render function. Render is really a very thin wrapper around the Drupal render and that's good for templates, you don't want to have too much pHp in there.

Summary 46.49

It's become a lot easier to change parts of the page, move things around and change things on a page by page basis
Hook page alter is the way you do that and we've improved performance on the way or at least given you to tools to improve your site's performance
Drupal core doesn't do a lot of #cache work because only you the site builder knows what's happening in the nodes down in the render arrays and what the cache keys might be for the modules you have installed so it's really a site builder thing not a Drupal core or contrib thing.

[Transcribed by Shreya Sanghani]

More recent presentation: http://sf2010.drupal.org/conference/sessions/page-render-drill-down-drupal-7