How the ‘F9’ filmmakers used camera arrays and bluescreen shoots to make fast action

Amongst an impressive array of stunts and practical effects filmmaking, the Fast & Furious movies have always made use of visual effects to help tell their stories. This might be to make vehicles—and characters—carry out extraordinary jumps, leaps and crashes that cannot be staged physically, to place actors inside cars during daring stunts, or to extend locations where action perhaps could not be filmed. A common practice in the movies also, for example, has been to piece together multiple elements for big action sequences like on location plates and bluescreen stage in-car or fighting scenes with the stars, and composite them together.

All of these kinds of things occurred during the making of F9, with production visual effects supervisor Peter Chiang from DNEG overseeing the effort of several VFX studios. The film would ultimately involve 2284 visual effects shots led by DNEG, with additional work from ILM, Stereo D, Lola VFX, Proof and an in-house team.

Here, in a wide-ranging interview with befores & afters, Chiang outlines the methodology for shooting car scenes, in particular describing how canera array plates for action shot backgrounds were captured, and the process for matching the real lighting when it came time to film actors on a bluescreen stage. We also discuss some specific shots, such as the car being pulled by the magnet through the shop window and that crazy rope swing moment.

Also, in a special bonus section of the interview, learn about a deleted shot from F9 that would have relied on a film scan of the original 35mm negative of the first movie featuring a scene with Vin Diesel and Paul Walker, with the scene seamlessly extended and expanded with visual effects.

The whisky shop shot

b&a: Let’s start with that magnet shot of the car being pulled through the whisky shop window in Edinburgh. I liked that [director] Justin Lin actually shared a making-of that very early on on Twitter showing the car on a rig and the motion control shoot. How did you approach this shot?

Peter Chiang: So that actual set was built on the backlot at Leavesden. We had a little bit of road, and we had the shop that was two-sided. We had to do it in two halves—the shot was actually split in two, because we were trying to respect the sunlight. We had one bit of road built with one half of the entrance on the right side of the street, and the exit on the left side of the street so that we could shoot in the same time of day. This meant that it would retain the same shadows and everything.

What made it a little more complicated was that we wanted to do a link up between two motion control moves. Ian Menzies oversaw the motion control. We had a device called The Bolt, it could do two metres a second. So it was a matter of mechanically working out the physicality of moving a fixed car in the location and then, also, linking that up with the motion control rig.

The way that special effects pushed the car was to use a lot of air rams and pulley systems. It wasn’t linked to the motion control system. So there was a lot of testing in order to sync the position of the car to where the camera would be at any specific point. Previsualization supervisor Alex Vegh started off by previs’ing it, and then we took over with a lot of the technical previs in order to get the consistent moves going.

We were working out the physicalities of the breakaway pieces, slowing the physicality of the car down, not to the point where it would impede the camera move, because Justin wanted the car to drive at the camera, the car to flip over, and then the camera to back pan as it moves through the window. Then we hit optimal acceleration, and from that we then pick up the camera on the second move, as it goes through the window, and then a truck drives along and it’s pulleyed or winched into the back of the van as the camera chases it and lets it go down the street.

We always thought that there was no way we could do it as a oner, so that’s why it was broken up into two moves. The middle bit would be CG, the end bit would be CG. The opening will be a bit of live with a bit of CG. DNEG ended up doing the comp in the end with all the CG hookups.

b&a: And like with all the Furious films, it feels like so much of it is, ‘Shoot what you can practically, which either stays as a partial part of the final shot, or serves as great reference.’ Here, because this car was pushed through something, did that help at least with DNEG seeing how much of a shop could be torn apart?

Peter Chiang: Absolutely. This is my first Fast with Justin. One of the things I said to him was that—and he agreed—we should always shoot a reference of something. So even the crazy things like the truck flipping over, we wanted to do it physically.

All the time, we were trying to push Alistair Williams, the special effects supervisor, to come up with ideas to physically represent these things, so that at least it’s based on some sort of reality. Of course, a lot of the times, these things are far beyond what any physical device could do. So it was a combo of looking at truck accidents that happen on roads, and landslides, and other things in order to gather reference. But wherever possible, we would always try and shoot something.

The approach to plate shoots

b&a: Just staying on car sequences for a minute, I’m always curious about when you’ve got these actors who need to feature in the cars and be in close-ups—we know they’re not driving the cars for those insane stunts, but what are the different techniques that end up getting used to mix and match shots, and put the actors in these cars, to get the right close ups, while making sure the backgrounds are correct? I’m really curious about the mixing and matching of real photography, blue screen photography, and digi-doubles and what else you could execute there.

Peter Chiang: Well, that’s a big part of the Fast & Furious films. You know, for some of the big sequences, we shot in Georgia, in Tbilisi, and we also went to Thailand. None of the actors went to those locations. They did shoot scenes in Edinburgh, and they did driving in Los Angeles as well. Right from the outset we said, ‘Okay, the actors aren’t going to go on this, so what are the tools that we’ve got in order to do that?’

For me, approaching this film, the plates were really, really important. What we ultimately used was a 360 degree camera array to shoot with and that would give us our background plates to composite our actors into. For example, if we just take the scene in Thailand, which has all those off-road vehicles.

Now, as you know, compositing-wise, we want to deliver to the 2D compositors static plates—as smooth a plates as possible—because then we try and match the movement of the plates in the bluescreen shoots. However, we also want to emulate what the terrain was in the location. So, we had encoders built into the vehicles, all the vehicles that were moving, so that they were time locked. If we had four cars driving along and Vin’s in one, and Roman’s in another, and Letty’s in another—well, she’s on the motorbike—we would need to shoot an array plate for every single one of those cars. This is because Justin, during the action sequence, would want to cut to Vin, then he’d want to cut to Roman, then he’d want to cut to Tej, then he’d want to cut to Letty.

So, we worked with Spiro Razatos on this, he’s the second unit director. Whenever we did a run, a setup, say we had three cars and a motorbike, I would then go in during that time of day in order to get the same lighting and keep three cars there and, say, have Vin’s car as the array vehicle. We would run the array vehicle with the other three cars. So that in the array, the motorbikes and the other cars would all be moving at the same rate and in that same formation. The stunt drivers would drive the array vehicles.

Now, the complication to that is, the array height is different for a Dodge Charger, to Tej’s vehicle, to the motorbike, to everything. And then a further complication to that is that the array plate needs to be stabilized. You can’t get a 360 degree, eight camera array stabilized. We had to split it up. I figured that most of the time we would be looking at the drivers. So we had five cameras at the back of a stabilized head, and then we had three cameras at the front that were stabilized. And then we had the shortest vehicle possible, because you want to try and link the cars up together so that they’re 360 degrees. And then we also designed the ability to be able to move those cameras to different heights in order to cover all of the vehicles.

b&a: Can you talk about how the shoot would work with these arrays?

Peter Chiang: On one pass of a setup, we needed to do four array plates all at that time of day with different heights to represent the different vehicles. And then added to that, I wanted to make sure that Steve Windon, the DP, had excellent lighting reference. The array was all ARRIs, but I had a fisheye set up with a RED HELIUM as well on top of the array vehicle with a 360 degree lens. This was so that I could then give a VR clip on an iPad to Steve on the bluescreen set, so that he could see where all the shadows were, and the lighting and everything.

Now, if you think about it, the pass of the camera, stabilized, would obviously have an overlap on the side cameras, a bit of a gap. And they would be stabilized separately, because obviously if you’re going downhill, one camera’s going to be higher than another. And so the stitch has to be slightly delayed in order to build up a 360. And then it all has to get put together. So instead of having to stitch hi-res plates, which would always take a long time, we always shot low-res video as a feed for that, so that it could be an automated system in order to stitch up the plate.

b&a: Right, so how did you manage the workflow in terms of sharing those plates?

Peter Chiang: This is how it worked: in post we would go, ‘What was that plate? We need Roman’s car for this shot. What was it?’ Now, added to that was then the stabilizing system. So although we were stabilizing the cameras, the encoders were encoding the actual bumps themselves. And what we did was, we then were able to take time code and take that exact movement of that car, and put it into the car on the bluescreen. Steve Windon had the VR for the lighting. He also knew, because we had a six axis gimbal rig, we could then plug in that move that Alistair Williams and his team had done, so that the car would move exactly as it had done in order to get the camera, so that we could then get the tracking information from that and then apply it to the stabilized plate in order to do the comp.

Every car array was encoded, and every plate was fed into a six axis gimbal move in order to do the move. It takes a while to get your head around it!

b&a: I wonder, is there an evolution of this approach that you see where you’d be able to use those array plates and set up an LED volume and shoot the actors in the cars on that?

Peter Chiang: No, it doesn’t work because of the terrain. Because if you think about it, a camera looking at a car is in sync with the car. And so therefore, the background behaves in a set way. We thought about the LED treatment. First of all, the lighting and the quality of it is okay, but it depends on the pitch of your LEDs. So, it was far easier to keep everything a little bit fluid.

Sometimes Justin would say, ‘Hey, you know that big bit of bump in the encoder? It’s falling right on my line and I don’t want that. So we had the ability to string up encoded moves and slightly edit the curves, so that the pitch of them weren’t quite right as represented. If it was too harsh, we could iron that out. We were recording all of this data, whatever we did, handing it over to the CG artists, and there they could match exactly what was done on set. So it gave more flexibility to be able to change it.

b&a: And in relation to backgrounds, did you need to augment what was shot or did you need to create virtual environments at all for car scenes?

Peter Chiang: We didn’t do a lot of alterations to the locations, although of course sometimes we made a CG location. For example, if you look at the end sequence at Tbilisi, we shot plates there, but really, because Justin had a very specific idea as to what the environment looked like, we were always doing set extensions. The local area around the cars, I would say a 100 feet, were probably real. And then the rest of it was CG that was based on another location. But it was all based on pretty much real stuff, photogrammetry scans.

b&a: What about scenes not with actors in the cars, but, say, fighting on top of them?

Peter Chiang: Well, when they’re fighting on top of the truck, we had to have an array plate that was 17 feet high, going along the road. And there, because we were on the road and we didn’t need to have such a stabilized system, I could then have the eight camera array. So I had a five camera, a three camera stabilized, and then I had an eight camera that wasn’t, that could go up on a column. So we also had to have two sets of those, because a 17 foot column that could drop down to an 11 foot column when Han and Mia fight on top of the armoury vehicle, that was okay for one. But that rig couldn’t get down low enough to do the four foot where Vin and Letty were riding in the Charger, in their car.

So, we had to have two vehicles. It was a pretty crazy process. In order for me to shoot efficiently, I had these vehicles all ready to go. I would liaise with the AD. They would do a pass with all the cars and everything. Then they would go back to the start, the stunt drivers. Stunt drivers would then get into the various vehicles. And then I would have all the camera rigs and everything ready. We’d just do one run, go back to the start, swap the cars around. While they were swapping the cars around, I’d lower the other rig. So then they would do the other cars. And then that car would be ready to do. I had probably half an hour to get my plates, because they wanted to shoot on. And then it was also important to do it in the half hour, because the lighting would change.

b&a: It felt like there was a really strong marrying up of plates, actors in cars on bluescreen etc—what were the particular challenges of making it all hang together?

Peter Chiang: In terms of shooting bluescreen of the characters in the cars, we also wanted to push that, so in the film, there are various scenes where we drift onto a car and ‘find’ the actors in the car. And they were just fantastic to do. So we would shoot wild plates on location, going into a stunt driver, and then we would shoot a MoCo move. We would track that shot, and then do a MoCo move on bluescreen, and put the actors in. And those were the ones that I found were really lovely to do. So I think in the opening sequence, in Thailand, we come up to find Tej, and then we pass Ramsey, and then we go past Ramsey, and then we’ll find Roman in the armoury truck, which was a really nice fluid thing that tied the actors to the locations.

Normally you’d cut to a wide, we’d do a CG face replacement, and then you cut to the actor. And we said, ‘Come on, let’s do it.’ Let’s do those moves and mix it up a bit.

For me, these arrays were something that needed physically working out because it was so integral to the Fast & Furious films, in terms of the complications of stabilized heads and giving the artists the material they need without restricting them with motion blur and having to try and work it out.

And it was important for Steve Windon to have the perfect lighting tools in order to shoot it so we could comp it. He would have a 360 sphere of any plate. So he could run it, and then look around wherever it was. Go to a frame, look around. So he had the perfect lighting tool on an iPad to light it. And we had the gimbals on the back lot, 360 with special effects so that we could turn it to whatever angle. So key light was back left, or front right, or whatever we would get the same true reflection.

So we were quite anal about getting those elements together, because there’s nothing worse than trying to flip a bluescreen to fit a plate. And that planning of all those elements with the DP, special effects, second unit, working out the encoding, the plates. All helped, each department chipped in to make it the shots that they are. And I’m proud of that.

CG cars

b&a: Obviously there are also CG cars and vehicles in the film. What are the things that are still hard about digital cars?

Peter Chiang: There’s so many things. I mean, we scanned all the cars. We take pictures of all the cars in different lighting conditions, neutral, everything, so that we can relight them, everything like that. But it’s the subtle differences in suspension and weight. And I think that’s something that we’re very conscious of, because in a lot of the sequences, and even in the end sequences, the certain actions of the cars didn’t work.

For example, sometimes the Marauder ended up being in the wrong place. There’s one particular shot where the two cars are running level, and if you see the shot, the original plate actually has a Marauder in it. The only car that you could see is Letty. Justin said, ‘Hey, I really like that shot, but we can’t have the Marauder in it.’ So we took the Marauder out, rebuilt the plate with the array plate, put a CG Vin in over the top of it and just controlled the movement just that little bit more. And there was a lot of choreographing of that sort going on.

Another moment, the first time Vin goes to engage the Marauder, that’s all CG. The first time the Supra goes in to engage, that’s CG, because they just couldn’t get the dynamics physically to work. Because if you think about it, you would jam, and then the front brakes would go, and it would judder, and it would smoke and everything. So best as they could was to ram it, and it didn’t look dynamic enough. So we ended up painting it all out and putting a CG car in.

The car swing

b&a: There’s another huge sequence that I wanted to ask you about, which is the car swing. How was that even approached?

Peter Chiang: Justin and Alex Vegh had worked out the mechanics of that. My first thing was, how are we going to make it look real? Originally, they just had the Tarzan swing coming up and it just leapt over the top and then landed. And I said, ‘Look, that just doesn’t work.’ One, the way that they’d shot that car, rolling, didn’t work with the way that the car was spinning. So I invented this flip so that it hits the ledge, and that way it could give me creative licence to turn the car to whatever angle it had landed when shot for real.

We looked at a lot of reference. The Blues Brothers dropped a car. So we used that as a reference. I think it’s a red car in the Blue Brothers that they dropped from a helicopter, physically. Sometimes too with previs, there’s a tendency to make it look faster, because you don’t want to hang onto a shot for that long. So we ended up slowing it down, looking at pendulum balls, and looking at all sorts of things to try and get the weight right. But at the same time, there’s always a fine line between making it too real and boring, and trying to keep it exciting.

b&a: We should also talk about going into space. What went into the space shots to make them as believable as possible?

Peter Chiang: We’ve had some great space movies. There’s great reference out there. And NASA, obviously, has fantastic reference of Earth. So we literally said, ‘It’s got to be NASA. It’s got to be NASA’s Earth.’ ILM did the rocket launch.

In terms of the car, we originally started with the Fiero straight into space, but it didn’t look quite right. So we thought, ‘Look, let’s do the ice thing. We’ve got to put the ice thing in there in order to give it something different.’ And then the movement of it, we looked at this great space shuttle footage of satellites docking. And if you look at them, it’s all pretty boring, because it moves at such a rate, and then changes, and then nothing else happens. So our shots are not Star Wars, but it’s not First Man, it just needed to just do a little bit extra. We’re a little bit of retro, a little bit of drifting.

b&a: What about when they were testing that rocket car at the, I think it’s an abandoned air force base in England, right?

Peter Chiang: It is. So, they shot that, and obviously they didn’t have the flame out the back and everything. So they had a remote driver in the car, and even in that car, Justin wanted a build-up to it, like dry ice and spinning wheels and all that. That spinning wheel shot is an all0CG shot, because they didn’t do the spinning wheel. So again, as the car comes out, and he’s controlling it, that’s a remote driver. But added to that is the dry ice, and the flames, and the flickering flame.

And then you cut into several CG shots of the wheel, and then the back of the car, getting ready. And then obviously all of the blast is CG. And then it’s a physical car getting blown up at the end of the runway. The plane that takes off early was also put in, and the runway’s extended. It was only three bunkers, and we needed six or nine. So it’s a set extension in order to clone it all. That was DNEG London.

b&a: Tell me also about those flashback scenes and the stock car crash. Was that mostly a CG sequence, or could they shoot something?

Peter Chiang: That was physically shot. I think they ended up with nine stock cars. In some instances, depending on the safety, they would only have three cars. Then for the skid, most of that’s done in CG. The view out the window of a spinning car is physical, but there’s only two cars doing it. So we would augment with three cars. And then they flipped certain Nascars, but the shots of when it leaves the ground were digital.

Justin was very specific about the action. If you watch the film carefully, there’s a white car that comes up, and as the car spins around, it gets clipped in a specific way, and then that gets launched. So again, it was shot, but all of the hit into the railings, the fire ball and everything, that was all CG. And then when it comes down to land, it’s the husk, that’s physical, but with enhanced bits; flame and debris. And then all the other Nascars are CG. And then the end wreck is obviously a physical wreck, but all the cars running around are CG. And the crowds were put in the stand. We had limited crowd, around 200 people, and then we had to do the CG cloning thing in order to get those right. That sequence was DNEG Montreal.

Bonus: a surprise deleted scene

b&a: Was there any de-ageing or digital work done for young Dom?

Peter Chiang: No, the only alteration we did was on young Dom was, we made his eye colors the same as Vin’s. That was the only thing. But, there was a time when we were working on a shot that started in the original Fast & Furious, and it’s the first time Paul Walker pulls up in the green car. Young Vin gets out and is berating him about getting on the gas too early or whatever. And the idea there was to pick up that move, as it was shot originally, which was shot on 35 mm film, and then pull back from that shot. So in other words, we’d go to different point of view of the same scene. Still with young Vin walking around. Paul Walker there as well. And then we pan to find the brother looking.

b&a: No way.

Peter Chiang: It was awesome. We went back to the original footage, I got all the original 35 mm film, looked at it all. We worked it all out that we could use projections from outtakes in order to build up young Vin. We did CG takeovers on certain bits that we couldn’t see. We had to build up the crowd. We tracked the move and then we did MoCo moves of crowd. We actually shot crowd dressed in the same clothing—in the original shot there’s crowds all around the cars. So if you’re changing the move, you have to wipe people in through the crowd, and they need to be dressed in the same costume and need to have the same lighting.

It would’ve been an awesome shot. But the only reason why we didn’t do it is that it would ‘throw’ people’s perception of young Vin, when they then saw the young Vin who had been cast in the other scenes. Because then they might be thinking, ‘Hold on a minute, there’s three Vins! There’s Vin looking like that, there’s Vin as he is, and then there’s young Vin. So Justin decided to take the shot out.

b&a: Oh wow. Do you think, possibly, the shots would’ve made it in if the approach to the young characters was de-ageing, which you didn’t do. But if that had been the approach, perhaps you probably could’ve done those shots, right? Because then it would be the same looking de-aged Vin?

Peter Chiang: We could’ve, yeah. Well, we looked at Captain America, and we looked at that and how that was done with body doubles, and the changes. The thing is, Vin is so unique.

b&a: Also, the actor who did play young Vin was amazing, I thought.

Peter Chiang: Yes. It was an awesome shot though, Ian, honestly.

b&a: I want to see it!

Peter Chiang: Ah, it was just so brilliant. Because we were scanning, at hi-res, the original 35 mm film, so that we could work out what projections we could do, where we could take over from the original camera. So when you suddenly cut to the shot, you think, ‘Hold on a minute, this is a shot out of Fast & Furious 1.’ So you’d think suddenly, ‘Cut to a Rob Cohen shot.’ And the car pulls up, Paul Walker gets out. Vin gets out and he’s going, ‘I had you. I had you.’ And then you change and you move round and you follow Vin, and then you pull away. It would’ve been a lovely shot.

I mean, I looked at the dailies from the day they shot that originally film. All second unit footage of just crowd. And we were literally grabbing people. We had names for every single person in the crowd. Hawaiian shirt girl. White T-shirt guy. And then we would look in the footage to try and find that. And then obviously going back to Universal’s archive to dig out that neg roll from 2001. It was so awesome.

It was the editing phase when we were putting it together. Justin, one session, he just said, ‘Peter, you’re going to hate me.’ And I said, ‘Oh no, this is bad news, isn’t it?’ He said, ‘Yeah, bad news. You know the lovely shot? It’s out.’ I said, ‘What? Why’s it out?’ And then he explained, he said, ‘Because there’s three Vins in the film.’