DNEG MADE A PROCEDURAL TOOL THAT LET ARTISTS CREATE BOMB-SHAPED HOLES IN BOSTON BUILDINGS FOR ‘THE LAST OF US’

In HBO’s The Last of Us, the Craig Mazin and Neil Druckmann-created series based on the Naughty Dog video game, the central characters make their way through several post-apocalyptic settings, some with bombed-out buildings and others displaying two decades of urban decay.

Production visual effects supervisor Alex Wang enlisted DNEG as one of the principal vendors on the series, and some of their key tasks included full scale environment builds for Boston and other cities.

Here, befores & afters chats to DNEG VFX supervisor Stephen James, DFX supervisor Nick Marshall and CG supervisor Melaina Mace about these cityscaping tasks, which included plenty of procedural approaches to buildings and vegetation. Plus, the team reveals a few Easter Eggs that eagle-eyed viewers should look out for in the episodes.

Building (a certain version of) Boston

The building of Boston’s destroyed cityscape began with a scout of the city and then an extensive week-long data capture, including aerial drone capture. “They shot in Calgary and Edmonton,” outlines Stephen James. “Our work was a blend of building out and capturing the Alberta city buildings. A lot of the time, what was captured in the plate was a building they wanted to keep and so we’d then just ‘destroy’ those buildings. Then we needed to build out extensive amounts of Boston itself to give it the feel and the iconic skyline that Boston is known for.”

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

DNEG utilized open source satellite data to provide a broad layout of Boston, as Melaina Mace explains. “We used OpenStreetMap data to build out the whole downtown area of Boston. We combined that with a photoscan from drone and aerial photography just to have a complete section of downtown that we could use in layout. We lined this up with our hero sections of Lidar that were scanned in Boston as well as Lidar from the sets in Edmonton and Calgary. We could then adjust the skyline as needed to get the composition we wanted per sequence or shot.”

Indeed, city-styling was incredibly necessary, given the mishmash of cities and the fact that Boston needed to look destroyed. That dictated an approach of starting with the real Boston, then, as Nick Marshall describes, “if we needed to make changes aesthetically from there and just artistically drive things to give us the type of layouts that we want to see, that was our approach. So, it was, ‘start from real’ and then deviate from it if it wasn’t giving you the right kind of narrative or aesthetic points that you wanted to hit.”

Dirtying and destroying buildings

Several methods were used to provide for the appropriate look of the bombed-out city buildings and surrounds. For CG buildings, DNEG started with initial base models and shading to match the real location. “Then we did another pass on top of that to add all of that decay so that we could actually render that full CG straight into the shots for our purposes,” says Marshall. “In other cases, it ended up being more of a 2D matte painting approach where we’d project out onto some geometry.”

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

When it came to the destruction itself, DNEG built some collapsed buildings for background insertion that could be re-purposed by rotating and placing them in different areas. Hero shots and iconic destroyed buildings required much more detail, including modeling them for FX destruction and even inserting furniture, wall interiors and building materials inside them.

“Our environments supervisor Adrien Lambert created some procedural tools for bombing, which were tools that let you draw up the shapes of bombs,” details James. “It would give you a base procedural destruction and it retained the volume so all the collapse would be at the base of the building. You’d get some interesting shapes, make sure that it had a nice natural feel to it, and then we could go in and model in and dress a lot more detail.”

“For example,” continues James, “we’d have to fracture exterior tiles on the buildings to reveal the cracking away of underlying layers. It’s been 20 years since the bombing happened, so you have a lot of compression and weight and sagging that’s happening over time, like the wood floors rotting away. And that was before we even got to the vegetation.”

So much ivy

Generating vegetation, which included a significant level of ivy growth and fungal growth, began with a procedural setup in Houdini. “This allowed our environment artists the ability to design and grow ivy over any sized object, from small set pieces to large scale buildings,” discusses Mace.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

“We had a variety of different ivy species, from traditional English Ivy to Devil’s Ivy to Bougainvillea, to help add some color for some of the springtime episodes. In the look-dev, we added a number of different attributes for age and density to the leaves and individual clusters, to help drive more color variation and also give comp the ability to adjust that later on, if needed. For other vegetation, we used a mix of SpeedTree assets and Megascans plant assets for vegetation scatters.”

A lot of DNEG’s real-world reference for ivy, specifically on larger buildings, was ultimately considered too evenly distributed. “It almost looked fake,” highlights Mace. “If you look at a building that’s completely covered in ivy, it grows very uniformly. The client was really looking for more of a wild unkempt design element to the whole city. It’s been taken back by nature, essentially.”

Dealing with the different seasons turned out to be a major challenge on the show, not just for the Boston scenes. “Spring, for example,” says James, “was not just about colors. It’s about that transformation and the impact of the story. A lot of consideration was put into those colors and the tones and what’s happening in the environment. It’s not always just about throwing green everywhere.”

“Even beyond that,” states Marshall, “if there’d been extensive growth, you would expect that some of that growth has died over time as well. So we would have underlying layers of dead growth that’s been taken over by more live growth that’s come over the top where it’s then started to take the light and the resources, and stuff underneath starts to die.”

Integrating the characters

The actors featured in Boston and other city scenes that DNEG worked on were captured in a few different ways. One was filming them in the city locations in Canada, another was on partial sets, and the final way was via some bluescreen/partial set pieces. In general, the intention was to always film in as much natural light as possible, so bluescreen was kept to a minimum.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

“One of our biggest sequences, called the Hair Salon, is where they first step out into the world in episode two,” says James. “It’s the first time Ellie is looking up and seeing this world, and there’s no bluescreens there. I don’t think they originally intended to go as big or as heavy with the destruction, and without a bluescreen it does need a lot more edge work, a lot more reinforcing and preparing edges – but do you really want to have these actors act in a blue box? There’s always that tricky choice to be made, but I think Alex Wang definitely made the right call of where to go to blue with some of our really full CG environments. Our amazing comp team on all this was led by Francesco Dell’Anna.”

The comp team, as well as the lighting team at DNEG, had one particular challenge in sometimes dealing with competing light sources for these city scenes. “On set,” recalls Marshall, “the light was changing so quickly that we had reflections bouncing around off of other buildings, because obviously they were shooting with real glass buildings around. In some cases, they could create very strong caustic bouncers, which means we had to do a lot of work on creating caustic light setups that would give us that kind of art directable look of that kind of bounce light.”

“Looking at some of our plates,” continues Marshall, “they almost looked kind of wrong to begin with because they had cross light directions and shadows that seemed to be at odds with each other. That was just because of the complexity of lighting in the real world, which is something that I think ultimately helps because it does feel more real when you actually get those little nuances in lighting rather than shooting everything on a bluescreen set.”

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

At one point, Joel, Ellie and Tess cross over from the rooftop of the Bostonian to another rooftop–known as the balustrade sequence. “For that one,” says Marshall, “we did quite extensive work to try and tie the whole sequence together. They were shooting over the course of the day. We’d have to supplement with artificial lights because the light was changing around. We would go from a fairly cloudy and overcast in some of the early shots, to then actually quite a strong almost evening-looking, sunset-y kind of vibe for the end of the sequence.”

“What we did do was retain some areas from the sets but actually play them as if those are just areas that are in cast shadows. There was some justification for that because there were some flags used on sets to try and guide the lighting. Instead of fighting the plates too much, we said, ‘Okay, well, let’s just darken those down a little further, cool them off a touch and then play those areas as if they’re in a cast shadow from some other structure offscreen and then we’ll key-light the environment around that.’”

Conversely, in some of the later shots, DNEG would dial it the other way around and push more of the sunlight into the plates. “We tried to create a sense that there was that little bit of passage of time,” says Marshall. “We used time-lapse HDRIs to find the right look for the sequence. And then what we actually did on this one, which is not something I’ve really done before, is start with a reference pass where we took the CG lighting and did a single frame lineup of our characters, rendered those gray shaded so that we could almost use our CG lighting as a reference for what that plate would look like if it had been shot in completely consistent lighting conditions on the day. And then we could use that to just slowly creep some of our plates towards our CG lighting. That gave us a really nice consistency over the sequence.”

Other cityscapes

In other episodes this season, DNEG was also called upon to craft views of Kansas City, Jackson, Wyoming and Salt Lake City. Like Boston, teams visited these other cities to capture reference and data. However, as James notes, “for Craig Mazin, it was important that we didn’t repeat what we had done every time we come to a city. They weren’t as much about heavy destruction and overgrowth as in Boston–it was more about the urban decay.”

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

The Jackson environment, for example, closes out this first season’s final episode, and required an extensive CG build, says Marshall. “I don’t know whether Alex was already thinking ahead to the next season potentially–it was never stated–but there was a desire to try and do that as a full CG build. So we built some specific Jackson buildings to create a main street and then use more of a procedural approach to fill out a lot of the suburbs and districts around that and theorize where our quarantine zone wall would’ve been built to keep that location safe.”

Did you catch these Easter Eggs?

Given DNEG’s environments and landscapes were so complex, did they get a chance to add any special Easter Eggs in their city scenes, maybe even as references to the original video game? Well, the team reveals, one of the biggest Easter Eggs the VFX studio was responsible for was not necessarily meant to be ‘hidden’.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

“But it was a bit hard to find,” says Mace. “I think it took viewers a few passes to see it. It’s the first time you see a clicker in the series, and it’s in the final shot of episode one. We went through a number of rounds of animation on different buildings to get the right placement and timing during the lightning strike so that you could actually see him.”

Marshall shares with befores & afters that there were also some VFX shots that the team particularly ‘geeked out’ on. “In the final episode, they made the decision to shoot in Canmore, Alberta, where they had a nice mountain location that matched fairly closely to the look of the Jackson Valley. One problem was, they only really had one nice-looking view. So they ended up shooting a lot of shots where Joel and Ellie are approaching and we’re looking away from Jackson Hole, and this was the exact same backdrop as when they’re then looking at it, ie. they’d cut back to back with these same views.”

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

Image Courtesy of DNEG © 2023 Home Box Office, Inc. All Rights Reserved.

“We had to do some nice little fiddly bits of work to creatively find ways to nudge the environments to be different enough that viewers wouldn’t go, ‘Hang on, weren’t they looking in that direction before? That’s the exact same environment.’ We had fun with finding ways to augment mountain lines and forests and river directions and things like that to knock your sense off that you were looking at the same environment twice.”

Finally, James suggests that “a lot of stuff is not caught by everyone yet. I encourage people to continue to watch and look out for little Easter Eggs throughout the episodes.”