[Update] Perfect Parking Spot – Shot 4: Crowds, compositing overhaul, and lots of rendering (Patreon)

Finished up with adding in crowds and now deep into the long process of rendering and re-rendering everything that's required for the final composite. In the process of doing all of that I've also completely overhauled how I'm handling compositing too. All in all, there's a lot to talk about this update.

Background crowds: addons, actions and the NLA editor

I've had the chance to look into some of the few crowd addons available for Blender now, and while some are more advanced than others, none really allow for simulating or faking what I'd call 'reactive' behaviour. They can scatter people around a scene, have them follow paths or even re-route them around obstacles but none (that I know of) will let you change what animation a character is performing during the course of a scene or shot.

You're basically locked into the one animation/behaviour for the whole thing. This isn't a problem for an establishing shot or one where the crowd is already running (or where you can use the camera to hide switching models) but for a static shot like this, where the action happens midway through, these addons aren't all that useful.

Instead I have to rely on using Blender's NLA (Non-Linear Action) Editor for handling crowds, which I've used together with models and animations imported in from Mixamo.

If you're not familiar with the NLA editor, in simple terms it functions like a video editor (think Premiere, Vegas, etc.) but for animation data instead of media files. Keyframed values instead of images or sound. It lets you store specific animations for a rigged character or object as contained clips (called actions) that you then chain together into sequences; blending them, layering them and even applying limited effects to them (modifiers).

It's an extremely powerful tool but one that's very clunky and cumbersome to use, which is why I rarely use it outside of dealing with background characters.

Background crowds: an archetypes approach to crowds

For this animation I'm building on the method I used for the micros in Rog's Sock Tease animation but expanded for a much bigger number of background characters.

I went through the shot and noted down when various events happened (e.g. when Rog teleports in, when the roof is destroyed, when his foot comes down, etc.) and then I set up 'archetype' characters which play through different sequences of animations based on what they're doing at each timestamp.

So for example: I'll have a character who starts the scene walking towards where Rog is going to teleport in, at frame(288) they stop to react to the appearance of Rog and play through some reaction animations till frame(439) at which point they turn and run the way they came:

Then I duplicate that character and offset that sequence a few times to create some variation for that archetype:

Repeat this process for a bunch more of these archetypes and I have a small library of characters acting out different routines that I can use to populate my scene via instancing. For this animation I set up about 20 of these archetypes (not counting the offset variants) covering things like:

Characters that end up walking off camera before Rog appears.

Characters that start off idling and then need to react afterwards.

Characters that start the scene moving, split up into categories based on their direction of travel compared to Rog (are they moving towards or away or parallel to him? Will they need to turn around to run or will they go to the left/right?).

Characters following a set path through the scene, which in this case I used for ones turning corners.

It's a long and finnicky process to set all of it up, but for the time being it's the only way I can create the illusion of crowds actually reacting to what's happening in the scene.

Compositing: overhauling compositing setup

Having finished with background crowds at the beginning of this month most of what I've been doing is rendering and re-rendering things as I've been figuring out more compositing stuff.

As a refresher, the main issue I was having with this shot is that it was too resource intensive to render all at once. So I opted to render it in isolated parts (e.g. the ground by itself, the vehicles by themselves, each of the collapsing buildings by themselves, etc.) and then combine those pieces back together via compositing.

The issue that follows on from there is now a bunch of objects are missing from each partial render, which in turn will produce inconsistencies between them if I render them with lighting and shadows enabled. To get around that I planned to render a special diffuse pass containing only the lighting/shadows for the full scene, which could then be layered back into the combined partial renders.

To get that pass I used material override, which lets you replace all the materials in a scene with one of your choice during rendering. That meant I didn't have to worry about the resource cost of loading/calculating all the shaders in my scene, but it also meant that I lost a lot of lighting and shadow detail. I was hoping to get some of that detail back in the final composite but I was unable to create a result I was happy with.

It was while I was thinking of a way to get around these limitations that I came back to the material override option. Specifically how inflexible it is. If I had a way to exempt certain objects from being overridden I could render them fully lit and with their own materials, while also keeping the rest of the objects active in the scene without their usual resource cost.

There's currently no way to selectively apply material override in Blender but there are ways to recreate a similar effect. In my case I have my various objects grouped up and saved into their own files, so it was easy enough to make duplicates of them with the materials manually remapped to a blank emission shader:

This allowed me to switch the 'no material' versions of various objects in and out as required for each partial render, allowing me to render them with shaders, lighting and shadows intact:

After that it's simple to layer up all the partial renders on top of each other to recreate the original image:

Compositing: adding reflections from Cycles

One important aspect that was notably missing from previous compositing attempts was reflections. EEVEE does support real-time screen-space reflections but they come with their own limitations and in this case I felt Cycles would be the better option.

Getting accurate reflections requires having most (if not all) of the scene visible with shaders intact, which brings us back to those performance/resource issues from earlier.

Luckily, Blender allows you to limit the resolution of all the image textures being used by shaders in a scene, so for the reflections I've rendered the scene with them all restricted down to 256px. The results are obviously much blurrier than the full-res texture version, but the reflective surfaces in this scene don't take up enough space for this to be all that noticeable (along with not being the focus of the shot):

Ideally I would've hidden all the non-reflective materials being used by these buildings from these passes, but they're currently being linked into the main file from separate ones that handle the destruction fx for each. Which is to say that each exists as an un-editable instance in the main file, so I needed to hop back into the others to make some custom masks:

You'll notice two of the three mask images are using red, green and blue instead of just black and white. Here I'm taking advantage of the fact that the image format I'm using stores colour in three channels (red, green and blue) and using that to store one mask inside each.

Blender allows you to separate out these colour channels during compositing, which lets me retrieve each mask and then use them to separately control how I want to handle compositing reflections for different reflective objects/materials (such as glass, metal and tiles):

As a side note on masks, when it comes to making custom ones you don't always have to do that yourself. Blender has a very useful compositing option called cryptomatte, which lets you create masks from specific materials and objects in your scene during compositing when enabled:

Unfortunately, in my experience cryptomatte seems to have trouble isolating transparent materials and instanced objects so I couldn't get much use out of it this time, but if you're interested in getting into compositing with Blender yourself it's something that's worth looking into.

Compositing: adding ambient occlusion from Cycles

For the longest time one of EEVEE's main shortcomings was that there wasn't much you could do to recreate the effects of light bouncing off objects/surfaces, outside of manually setting up bounce lights, leaving you with flat uniform looking shadows. Now newer versions of the engine include limited forms of raytracing to handle just that, but yet again this scene is too resource intensive to make good use of any of that.

Instead I decided to fall back on a well worn technique for adding more depth to shadows: Ambient Occlusion. It's a lighting effect that attempts to recreate the soft shadows found in the corners of objects where ambient light can't reach. Both EEVEE and Cycles can render ambient occlusion passes for scenes, but Cycles produces better results so I opted to use it instead:

Ambient occlusion shouldn't be visible in direct light, so I've also created a combined shadow pass for the shot and then used the inverse as a mask for where the ambient occlusion should be mixed in:

What’s coming up next

More rendering to come and a lot of it. There's one more reflection pass that needs doing and one that needs re-rendering. The EEVEE partial renders also need re-rendering at higher samples and then finally all the smoke from the previous update all needs re-rendering too.

I'm basically aiming to have the final render version of this shot all finished up and composited together as a dry run for compositing the rest of this animation.

All in all, this is going to take a while. If all goes smoothly I should be done with the reflections and EEVEE re-rendering by the end of next week. Rendering time for the smoke is harder to estimate (since it varies from frame to frame based on its changing density) but I'm going to assume it will push things through into the beginning of March.

As mentioned in my previous update, if rendering smoke seems like it's going to drag on I'll start alternating between doing rendering and working on other shots (whether for this animation or Sweaty Summertime Shrinking).