Motion Graphics

Multi pass and motion control: re-visiting the VFX of ‘The Fifth Element’

Luc Besson’s The Fifth Element is now 25 years old. Back in 1997, the visual effects for the film were realized with a masterful combination of motion control miniatures, CG, digital compositing and effects simulations by Digital Domain.

Perhaps most memorable are views of a future New York, complete with flying cars and a mass of new and old skyscrapers. The film was one of Digital Domain’s huge miniature shows released that year – the others being Dante’s Peak and Titanic – while also heralding the fast-moving world of CGI in the movies.

Here’s my original interview (first published at vfxblog) with The Fifth Element’s visual effects supervisor Mark Stetson, who re-visits the work, both miniature and digital.

This was your first visual effects supervisor role – how daunting was that for you?

Mark Stetson: I wasn’t afraid of the size of it. I didn’t think it was huge at the time. I mean, it was sort of standard tent pole-ish at the time and I was confident that I could do that, but it was my first one and there was a ton I had to learn, especially about digital visual effects. And I was very supported by Digital Domain. It was Digital Domain 1.0 back then, and they really gave me a great team. It was a great experience all around.

Prior to that you’d worked on things like Blade Runner and Hudsucker Proxy and Waterworld. You’d come from miniatures. Could you give me a quick background of what you felt like your area of expertise was?

Mark Stetson: I sort of came up quickly through the ranks in miniature effects and I think Blade Runner was my third or fourth picture. And running the model shop for Doug Trumbull was a huge deal in my world and in the visual effects world at the time. And so what I found is – I stayed with that for ten or fifteen years – was I really ended up fronting visual effects projects in the sense that model construction needed such a lead time that I really needed to sort of figure out the scope of the work. And oftentimes I was the first person doing a breakdown on a film: what was going to be miniature, what was going to be matte painting, what was gonna be comp, what elements we needed, stuff like that.

And so I had plenty of support in that regard but oftentimes I was the first person to seriously look at a project. So that all led to it not being an unusual transition to go into overall visual effects supervision. At least that was my foundational confidence, shall we say, my confidence in that sense. And I of course had tons to learn, but it’s a good place to start. Digital Domain was very supportive. I was actually kind of shocked when they handed me that project. I knew nothing about Luc Besson really and my first job was to read the script, get on a plane, fly to London and meet him. And let him bless me for the job.

And it was all pretty wild to start. I didn’t know what to make of the script, didn’t know whether to read it as a comedy or a drama, and had plenty of questions about that. But just sort of embedding myself with the film crew was a great experience, a great experience.

What do you remember about those early days of meeting Luc and going through the script, and what his view and your view was on how some of these sequences would be done?

Mark Stetson: Well, Luc was very impressed with my credentials, especially Blade Runner. And was happy to take me on almost based on that. Really he wanted to make a deal with Digital Domain and he wanted his picture to be done there. It was that more than it was me, and I was very lucky to become part of that project and part of that world. He had some pretty clear ideas and communicated them very clearly how he wanted to shoot things. He has a pretty unusual style, especially then. If you looked at his films then, he explained to me, ‘You know Mark I don’t want to do these ‘fancy panning around and seeing the whole world shots’. I’d much rather set a camera looking down a street, having a cab rush towards me, and cut as it passes by, and then cut to a reverse of it passing by, and construct my film that way.’ So there’s a lot of single centered one-point perspective shots in the movies, and his prior movies, too, if you look at them.

I remember there was one shot late in post in the cab chase we were trying to figure out where his sort of one-point perspective style just wasn’t working for the shot, where it was a shot of the cab after escaping the police and Bruce Willis relaxing for a moment when he enters an intersection and there’s a whole squadron of police cars there laid out waiting to shoot at him as he goes through. And so Luc had originally storyboarded it as if it was a tracking shot straight on the cab and you track with it as it enters the intersection, and then as Bruce Willis or Korben Dallas enters the intersection all of the squadron was there. But it didn’t give that moment of audience anticipation for seeing the cars and seeing him enter the trap.

So we had to really work hard to sell Luc on the idea of leading the cab, and then panning ahead to the police cars and then letting him enter that perspective. And I almost felt like I was betraying Luc asking him to do it that way. But once he saw it he understood what we were trying to do and he appreciated the shot. The shot turned out great.

With that experience and with Luc so clearly conveying how he wanted to construct his films, it made a real impression on me and it gave me a real understanding of filmmaker style, that one little exchange. Wes Anderson does that all the time, where he’ll start on a one point and then pan to another one point. And it cracks me up every time I see it in his films, half because it’s funny in his films and half because I think of Luc.

So, it’s 1995-96, I’m guessing, there’s been a lot of big advances in digital visual effects, but miniatures are still being used widely. Was this always imagined as a miniatures show with some new innovations in digital work? I mean, is that how it went down in Luc’s head and your head?

Mark Stetson: Yes, pretty much. We planned it out with miniatures and previs, and the art department in England was always planning to design the miniatures, and I sort of took that away from them and said, ‘No we want to design and build these in LA because we’ll be able to control them better and shoot them better.’ I had Bill Neil as my visual effects DP and he’s just wonderful. He wasn’t in it the whole time but we didn’t want to extend our stay there, and in fact the miniature shoot ended up being the same number of shoot days as the production was, so it would have been a tremendous delay in us getting back to work at Digital Domain trying to get back to shoot the miniatures in post in England.

So, that was the supposition. I had done a lot of work with big miniature cities, I’d built New York for several projects, and we were planning to do much of it with miniature buildings and scene extensions with matte paintings and other 2D techniques. So Luc’s shooting style sort of lent itself to that because we could plan out the paintings in a straightforward way and they were fairly static much of the time. There were a few shots that we ended up with CG cityscapes, when he dove down into the fog that scene of diving down through the layers of the city that was all digital there.

We built miniatures for the cab and the cop cars and some of the support vehicles, but we also designed a little squadron of support background vehicles that were always just digital, part of our so-called digital traffic system. And then we started experimenting with digital versions of the miniatures, and there was one shot after Leeloo escapes from the lab and dives off the ledge, there’s one shot where the cop car corkscrews down and follows her down, and that’s a digital shot. Nowadays you’d call it an Easter Egg, just something for us to try to see if anybody noticed that it was not the miniature, you know? We got away with it, it worked out nicely.

Tell me about the timeline. The film was shooting in London, but did principal photography happen completely and then the miniatures shoot happen, or was that done concurrently?

Mark Stetson: Well, I started at DD in July ’95, and I was handed that script and flew to London then. And I made a few trips back and forth in ’95 during prep, meeting Dan Weil and his art department, meeting with Luc, and meeting with Sylvie Landra the editor, as the project was prepping. And I flew back to LA with some designs in hand. We tried to get some things going in terms of miniature design and construction. The Mondoshawan ship we got an early start on, for instance. So we planned out the miniature crew and then went back in February I think of ’96 to start shooting.

And we started actually in Mauritania for the location shoot for the desert scenes. That was an amazing adventure all by itself. I think we were nine days in Mauritania, I don’t remember how many of them were shooting days, but it was out in the middle of the desert a hundred miles from the nearest town and we flew out there with helicopters and light planes, and landed the light planes right on the desert floor on a plateau near the shooting site. There’s a squadron of, a little fleet of pickup trucks and Land Rovers, and I think one nice Toyota. I think there was one Range Rover and one nice Toyota Land Cruiser. Everything else was like Hilux trucks and stuff.

And it was amazing, just amazing. We slept in Bedouin tents and it was an amazing experience. We had a big Bedouin dining tent that the crew would gather in, and they were all classic style, pointed top, four sided, three sides with big flaps buried in the sand and the fourth open side downwind. They’d lay down carpets inside, it was a wonderful way to live for nine days. And I felt guilty every time I used water because I knew it had all been trucked in. And there was a chef there, a French chef cooking for everybody and he had a little like a kitchen stove, like a white enamel kitchen stove with four burners on the top with a gas powering it. And he cooked for the whole crew on that thing, it’s so funny.

But, anyway, we went straight back to London after that and started shooting. Shot through until about Labor Day I think ’96, and during that time Neils Neilson built up the miniatures crew at Digital Domain. That model shop was equipped mostly with equipment from my old shop, Stetson Visual Services. I had closed my business before I started at DD. So they had the cabs and cop cars and the vertical subway, and a number of the buildings under the Mondoshawan Ship.

I had made at least one trip back during production to just check in on stuff and convey more information and review the material. So we started shooting miniatures after we wrapped production, so somewhere in May – June ’96, and I think it was a 105 day shooting schedule and a 105 day miniature shoot schedule too.

Can you talk a little more about how Digital Domain was generating these miniatures, or building them based off the art department’s artwork?

Mark Stetson: When Luc started the project, he had a storyboard artist that he had worked with, who storyboarded the entire cab chase and several of the sequences in the film. So we had that as a start. And then this was very early days for previs but we built a little previs team in London while we were shooting. And Karen Goulekas was my digital effects supe and she put together this little team. There was only about seven of us from DD in London and I think three of those were the previs team.

So they started building buildings and they’re very simple box buildings with black and white zebra stripes on them representing the floor height so we can get a sense of scale out of them, and the cop car and the cab and all the background traffic, they were all just little lozenges, very you know, no color maps on them, no resolution to speak of. I think we might have color coded them but they didn’t look at all like cars. They were just objects.

And I can’t remember what we were working in, maybe Prisms, it was kind of rough to get the whole thing set up. But then we started animating to the boards, and when we were ready Luc came in to start reviewing some of the shots. And we were so happy that he didn’t talk about style or finish quality or anything, he just started directing the shots. He started doing his job right away like he’d been doing it all the time.

And, this is a little aside, but it came up in a conversation I had recently with Luc’s visual effects producer Sophie Leclerc on Valerian. She said Luc could direct the previs because he had the whole movie in his head, and then she did a funny thing. We had lunch together about a month or so ago and she did a funny thing where she said she’d asked him a question about story or about shots or something like that, and he’d lift his eyes like this and think about it for a second and then come down and give her a straight answer, and that was it, that was the answer. And that’s the way he always was with me too. And to me he’s looking up and playing back the movie in his head and just seeing how the cut works with your shot in it. He knew the movie, you know he had the movie in his head, and he was just executing what was already in his head. So that was neat, that was really cool.
The shots, especially the miniatures, really seemed to feel part of that futuristic world of The Fifth Element – it wasn’t too futuristic, I mean. They were as grimy and gritty as the sets. I felt the same way in Blade Runner and Hudsucker Proxy. Can you talk about that side of things?

Mark Stetson: Well, I have a very clear understanding of your question and the answer is also very clear to me: production design. If you think about the production designers on those movies they’re all truly great production designers. Larry Paull, David Snyder, and Syd Mead together as essentially partner production designers on Blade Runner was an incredibly powerful combination of design and vision. Dennis Gassner on Hudsucker Proxy is just an amazing designer, always has been. Everything he does looks gorgeous and beautiful. And Dan Weil on Fifth Element – he was terrific, did a great job.

So on Fifth Element in particular Dan and I became very close during that project, we spent a lot of time together, and he was always talking about the movie and about the way Luc thought. And one thing he had, one thing he gave me, besides giving me sort of construction drawings and progress for the miniatures that they had begun and stopped when I told them that we really wanted to take that over in LA, gave me a pile of those, but he also gave me this bible. Luc had had a set of designers, a group of designers working on Fifth Element for about a year when it was in development at Warner Brothers, and they generated hundreds or thousands of illustrations for the movie.

And Dan gave me a collection of about a hundred of them that had to do with all aspects of the film. So most of what you see there is evoked from those illustrations, and then to think about the illustrative power on that thing. He had Moebius drawing for it, and then Jean-Claude Mézières who drew Valerian and, you know, the Valerian graphic novels are his. And Luc just hired him to do the movie. Instead of trying to rip him off he just hired him to do Fifth Element and much of the cab chase is straight from Valerian. And the illustrations he did for the cab chase are beautiful. And it’s all these crazy Baroque decorations of the city, it’s all the depths, all the perspectives, all of the jobs that I’d done before that trying to do miniatures for New York.

We could never really show shots all the way down the avenues because we just couldn’t build that far on a miniature stage. So that was the first thing we had to do, was figure out how we were going to do that because we knew we needed to, and Luc’s one point perspective, that’s how all the shots are gonna be constructed. So, you know, we used the illustrations as a guide and we started figuring out how to paint the extensions, and it was really great. And I had enough model experience personally and I had worked with Neils before on projects and he put together a great team that I helped him cherry pick.

Actually, what was the methodology in building the city, filming it with that one-point perspective, but also filling it with miniature cars? Can you take me through how you actually thought you’d build up these shots?

Mark Stetson: Well, we built an inventory of buildings that would be recycled building to building, and then scene by scene we tried to come up with some accents in the foreground. You know, what’s going to really make this neighborhood? What’s gonna make it different, what’s gonna make it stand out? So when Leeloo comes out on the ledge of course it’s the lab in the foreground matching the set design of the ledge in the foreground and we had an illustration of that from Dan Weil and the team. This one was a Mézières one, of the buildings across the street and the arch bridge across the street, all that. So we knew that those were special details we’re gonna use for that scene. And in fact much of the design of the overall set was based on that first setup because it was such an important part of the shot.

And so Neils had a crew in the shop working, and then a crew on set working. And to keep the shoot going generally we turned around the set for a new setup every day, and Bill Neil was shooting away we were attempting to shoot, we attempted and failed to get motion control files from previs into the motion control programs for that film. We succeeded on the next one. We gave it a really good try on Fifth Element and it kind of failed, but ultimately just programmed shot by shot in that regard. But when you start the next setup we’d already discussed with Neils and the team and Digital Domain’s art directors Ira Gilford and Ron Gress and the art department what we’re gonna change to accent the next scene.

The down shots there was this building across the street with this whole mirror finish on the front of it. There was an Italian neighborhood with all the laundry on the lines. There was the neighborhood where the President showed up with his limo at the end of the movie that had a much more palatial feel to it. Every scene we tried to do something a little bit different. The McDonalds had a whole different feel to it. And we were also sort of mapping out where we were in the levels of the city too. You dove down in the sequence it was a little darker down in the next part.

So the model crew would just go to town and re-detail, re-paint, re-compose the buildings to make it work. It was a painter, or actually a paint team, Jamal Fort led the set work. And Ron Gress was like the best miniature painter in Hollywood at that time, so if there’s ever a question, you know, Ron would know how to fix it. But Jamal and I think his painter was Laura did a fantastic job just sort of recreating these setups day by day.

Was there a lot of confidence in digital compositing at that point, seeing as you would have the city, but then layers and layers of the traffic shooting?

Mark Stetson: There was. Compositing was of course a really big part of that show. Jonathan Egstad and Bryan Grill were the compositing supervisors. NUKE was being written as we worked and it was an in-house program at that time. Now it’s an industry standard. The transformation to digital compositing had happened in the prior five years or so, so during the life of Stetson Visual Services which was ’89 to ’94 or the beginning of ’95 we watched all our clients go from optical compositing to digital compositing. And it affected our work somewhat but not as much of course as doing CG work.

But digital compositing was pretty much an industry standard at that point. Everyone was writing their own software, there wasn’t common off the shelf software so much. There were a few programs that people were using but I can’t even remember what they were, but you know like Digital Domain, Sony Imageworks, Rhythm & Hues all had very powerful compositing programs that they were writing and updating, maintaining themselves.