3d graphic design, 3d animation studios

Revealed: how they brought back Luke Skywalker for ‘The Mandalorian’

The de-ageing work was showcased in the latest ‘Disney Gallery: Star Wars: The Mandalorian’ ep, which also looked at the deep fake explorations done by the team.

It stunned viewers, and now the visual effects secrets behind that s2 finale of The Mandalorian–where a Young Luke Skywalker returned to care for Baby Yoda–have been revealed in the latest episode of Disney Gallery: Star Wars: The Mandalorian on Disney+.

Ultimately, it came down to Mark Hamill performing the role on set, with a stand-in actor also used, and Lola VFX’s de-ageing techniques adopted to produce the final result of a Young Luke. What was fascinating to learn, as well, was ILM’s early explorations of deep fake techniques as a possible avenue to achieve the desired look.

I’d highly recommend watching the episode for very specific details, but here’s a quick summary of how the de-ageing was achieved:

1. Through a great combination of on-camera interviews–including many VFX reps such as Richard Bluff, John Knoll, Abbigail Keller, Landis Fields, Trent Claus and others–we learn that the filmmakers first looked at the different techniques available to do de-ageing work. This included doing completely CG characters like Grand Moff Tarkin in Rogue One, a digital ‘airbrushing’ approach, and deep fakes (at the time, of course, one of the very new methodologies available).

2. Landis Fields at ILM was tasked with exploring the possibility of deep fakes to achieve the shots, and he carried out extensive R&D, referenced original 4K scans of the past Star Wars films with a younger Mark Hamill as Luke, of course, as well as past interviews and photographs for ‘training data’.

3. Ultimately, owing to where deep fake tech was at during production, the filmmakers still relied on ‘traditional’ de-ageing techniques, in this case those employed by Lola VFX that you will have seen in several Marvel and other films.

4. Lola’s Trent Claus explained their approach: on set, both Hamill and an incredibly well-cast age-appropriate body (or picture) double, Max, performed the scenes. The actors would then be taken to Lola’s proprietary lighting rig–dubbed ‘the Egg’–which is made up of programmable LED panels that can be animated and controlled to replicate the correct lighting, to acquire high-res facial textures.

5. Lola’s technique to do final de-ageing work–although this was not discussed in detail in the episode–has traditionally been to work with all these elements mentioned above, as well as past footage, photographs, and sometimes cyberscans of the actors, and then ‘craft’ the de-aged character with 2D compositing methods, largely in Flame. It’s incredible work that they do, paying close attention to how faces change over time. They also had to take note of the different ‘likenesses’ Luke seems to have in different scenes in the Original Star Wars Trilogy (an issue brought up in the episode). I am not absolutely clear about the direct role of the deepfake R&D in the final implementation, but it appears the deepfake tests and approach served as a baseline and ground truth in the process, and perhaps even further than that. The deepfake imagery shown during the ep is pretty cool to see.

I really loved this Gallery episode. We hear from many players, and there’s just incredible behind the scenes footage. I also enjoyed mentions of the ethics of deep fakes, and the note from John Favreau that Young Luke’s voice is entirely synthesized using a tool called Respeecher as well as via the artistry of Skywalker Sound.

Congratulations to all the artists involved in making these super-fun moments in the s2 finale possible.

Source-https://beforesandafters.com/2021/08/25/revealed-how-they-brought-back-luke-skywalker-for-the-mandalorian/