‘Breaking Down The Plate’ For Avengers : Endgame – Inside The VFX Animation Process

Getting to a final shot or sequence in visual effects takes a lot of steps and a lot of artists. There’s the script, concepts, storyboards, previs, stuntvis, live-action shooting, postvis, editing and the final visual effects – not necessarily in that order and occasionally not including some of those steps at all.

One step in the process that is sometimes not often discussed is the concept of ‘breaking down the plate,’ that is, when a visual effects studio receives from production a locked or edited sequence, or plates, to start work on. It’s at this point that the VFX studio needs to identify what work will be involved, all the way from plate preparation to final animation and lighting, and begin distributing tasks to artists.

To find out how that worked on one particular show – Marvel’s Avengers: Endgame – VFX Voice talked to Weta Digital Animation Supervisor Sidney Kombo-Kintombo [who requested that he be referred to by his first name]. Here he breaks down, from an animation point of view, aspects of the final spectacular battle from that film, where live-action plates, motion capture and significant digital visual effects were all involved.

Sidney Kombo-Kintombo, Animation Supervisor, Weta Digital.

A greenscreen plate for a scene in which the Avengers begin to assemble. (All images copyright © Marvel 2019.)

Weta Digital’s final composite.

“These films are gigantic machines. There was some incredible previs and planning, but no one expects everything to be there when we have such a big sequence. So we were able to jump in and work on that moment. We did have some previs for that [Women of Marvel] scene, but they wanted something bigger. The thing is, we didn’t have much time.”

—Sidney Kombo-Kintombo, Animation Supervisor, Weta Digital

STARTING THE PROCESS

Working in conjunction with Production Visual Effects Supervisor Dan Deleeuw and Weta Digital Visual Effects Supervisor Matt Aitken, Sidney’s process after receiving turnover from Marvel on end battle sequence shots began with piecing together the various plates, previs, postvis and notes received.

“We call this the ‘breaking down the plate’ stage,” explains Sidney. “We have the plates, but now we need to do the turnover notes for these plates for the animation team.”

The Animation Supervisor credits his Production Manager, Eleanor Morris, for helping him to identify and catalog all the different things that now needed to happen.

“We would look at the plates and say, ‘Well, we need this character to be matchmoved from frame one to frame 10. We need this character to be painted out from this frame to this frame. We need this guy to be removed or painted out. Or, OK, we’re going to put the CG character right here so that we don’t have to reconstruct the plate and so on and so forth.’ Eleanor would take notes of what I’m saying when I look at the plates as we come up with a plan to make the shots work.”

It’s at this point, too, that Sidney did draw-overs on the plates themselves. The idea here is to give a quick visual guide of what needed to be done to the plates, either in preparation or in terms of actual layout and animation. “Draw-overs help the matchmove, roto and paint teams identify what they need to concentrate their effort on,” he says.

The resulting notes could be a mix of simple items and much more complex descriptions. For example, recalls Sidney, the notes for the shot when Captain America says, ‘Avengers, assemble,’ took a lengthy 10 minutes to read from beginning to end.

On-set plate. Postvis would be carried out first, with Weta Digital then called upon to produce the final elements.

The final shot with animation and extensive environment work.

“I made a list of the elements that I thought we were going to need to shoot [to add to scenes]. I sent that list to Marvel identifying the reasons we needed new capture or needed to re-shoot capture that they had already done. And they never pushed back on anything – it’s an extra expense for them, but they trusted us to know that we needed more.”

—Sidney Kombo-Kintombo, Animation Supervisor, Weta Digital

PLUS’ING THE SHOTS

It’s also at the breaking-down-the-plate stage that Sidney identified where additional footage and, in this case, motion capture might be required. There were so many characters and fighting moments involved in that final battle, of course, that it was deemed some additional capture would help add to the scenes.

“I made a list of the elements that I thought we were going to need to shoot,” Sidney describes. “I sent that list to Marvel identifying the reasons we needed new capture or needed to re-shoot capture that they had already done. And they never pushed back on anything – it’s an extra expense for them, but they trusted us to know that we needed more.”

Sidney had, in particular, identified that further action was needed for the mid-ground areas of the battle. He put together some story points focusing on how certain heroes would take on certain villains and show the battle progressing, and made notes on that. This ultimately became the basis for a new shoot with motion-captured stunt performers on the shooting stages in Atlanta, which was attended by Aitken.

Adds Sidney: “The story we were trying to tell was that, at the beginning, the heroes are running after the villains – they are beating them. But the more the battle progresses, the more we see that all the villains are still outnumbering the heroes. For one kill from the heroes, you have two kills from the villains.”

STORYTELLERS

There were other ways that Weta Digital plus’d shots in the battle, too, including helping to devise key moments like the ‘Women of Marvel’ moment with fresh previs. “These films are gigantic machines,” acknowledges Sidney. “There was some incredible previs and planning, but no one expects everything to be there when we have such a big sequence. So we were able to jump in and work on that moment.

“We did have some previs for that scene, but they wanted something bigger,” Sidney continues. “The thing is, we didn’t have much time. We had literally one day to previs this entire thing for the Women of Marvel scene. Deleeuw mentioned it to us on a Thursday evening and on Friday morning I was brainstorming the sequence. Friday evening I pitched it to Matt Aitken. He was onboard with some more notes, and then we called Dan and he added his own notes and we came up with a solid plan.”

A previs effort at Weta Digital then went into overdrive to map out the story points in rough animation, as well as incorporating existing plates and new elements to help make the final shots possible. This was approved by Marvel, enabling Weta to then produce the final shots.

All of this, of course, is done in the aid of telling a bigger and better story, something Sidney always has on his mind in breaking down plates, working on previs and in overseeing final animation. “One Dan Deleeuw comment that came back to us,” he notes, “was something like, ‘When you guys get a shot, it comes out of Weta Digital very cinematic.’ That was a huge compliment to us as we worked on this very huge project.”

A wide view of Weta Digital’s portal opening scene in Endgame’s final clash.

“One Dan Deleeuw comment that came back to us was something like, ‘When you guys get a shot, it comes out of Weta Digital very cinematic.’ That was a huge compliment to us as we worked on this very huge project.”

—Sidney Kombo-Kintombo, Animation Supervisor, Weta Digital

Captain Marvel tackles Thanos.

“We would look at the plates and say, ‘Well, we need this character to be matchmoved from frame one to frame 10. We need this character to be painted out from this frame to this frame. We need this guy to be removed or painted out. Or, OK, we’re going to put the CG character right here so that we don’t have to reconstruct the plate and so on and so forth.’ [Production Manager] Eleanor [Morris] would take notes of what I’m saying when I look at the plates as we come up with a plan to make the shots work.”

—Sidney Kombo-Kintombo, Animation Supervisor, Weta Digital