The technology that will revolutionize the art of filmmaking ft. The Mandalorian
Let’s look at the technology used in the filming of THE MANDALORIAN
Green screen is a Filmmaking necessity. But soon in the future, it’s going to be replaced.
But why?
Having a green screen is arduous
One of the hardest things in filmmaking is for an actor being ready as a character while surrounded by huge walls and soft foam blocks in green that will be composited later and the characters have motion-capture dots on their face. Moreover, everything filmed within green screens will have green reflections that will be colour corrected in post-production.
Of course, there have been done some advances like having instantly switching backgrounds with CG backgrounds and characters into monitors. It does help in composition but it’s not that good yet. Lots of filmmakers have tried to just show an image behind the actors without greenscreen using single panels or projections. And that may work totally fine if you have to fake a shot or fake a certain view without actually going there but the problem starts when you don’t want to have a static shot but a moving shot. As when a camera moves it immediately portrays clearly that the background is a flat image.
There’s a technology that might just solve this problem.
ILM’s StageCraft System
Meet the ‘Volume’
Mentioned as StageCraft by the visual effects company named Industrial Light & Magic, it’s 20 feet tall, 270 degrees around and 75 feet across and is the largest and most advanced virtual environment for filmmaking ever made.
Industrial Light & Magic and Epic Games(makers of the Unreal Engine), together with the production technology of NVIDIA and others in collaboration with Jon Favreau’s, The Mandalorian.
Volume and it’s working
In filmmaking, a “Volume” basically refers to space where compositing takes place, which is then turned into a big set where actors play their roles. And the problem that it fixes is that it’s NOT STATIC. The walls are a set of huge LED screens as we might have seen in concerts but this is bigger than any of them and most importantly is smarter.
The creativity or innovation in StageCraft is that not only the image shown in the screen will be live generated in 3D powered by strong GPUs, but the 3D image is directly connected by the movements and the settings of the camera. When the camera is moved to a certain angle, the image precisely moves as well that makes it as if it’s a real scene.
Moreover, this enables motion-tracked cameras to achieve traditional cinematic shots like the Parallax effect where an object in the foreground advances at a different speed than the background that intensifies the illusion of filming at an actual location.
This was quite difficult to achieve as the camera must send it’s real-time position to a High-End PCs because setups like this run on Unreal Engine(it was used to build the virtual sets) which should take the environment and render it accordingly with the changes like lighting, perspectives, depth and distortion. All these should be so fast that all the changes can be played on the walls instantly. Moreover, half of the scenes in The Mandalorian were shot with this system, no one had any idea if this was done any differently than with a green screen.
Expensive?
StageCraft is obviously probably one of the most sophisticated, expensive and complex production system ever used. But it’s worth it as it pays back in different kinds of benefits. First of all, it basically ends the system of on-location shooting which is totally time-dependent and time-consuming and expensive. Instead of going to a Cold desert to get the wide-angle shots to create drama in the scenes, the team can just build a cold desert set and put that on the screen behind the actors. Or this can be also achieved by sending a team to capture the essential shots in High Definition 3D and then use it as a virtual background.
Moreover, the 2nd option adds another benefit that is the Reshoots become way too easier. If certain changes are made after the shoot in the dialogues or anything else went wrong rather than going to the location again by giving huge loads of money for clearance, you can just play the footages on the LED walls and ask the actors to act again with the same lighting and props.
This saves a serious amount of time in post-production as the VFX team doesn’t have to colour correct every reflective surface and do all sorts of minor compositions. Lighting is another aspect where this system helps that is the bright LED system gives a lot of illumination and because of this it provides realistic colours and reflections on props and actors which was never possible with green screens and were added in post. And the main character in, The Mandalorian had a reflective armour so the lighting on it had to be perfect which The Volume took care of easily and which couldn’t have been possibly achieved by green screens.
Although there’s one problem. At 20 feet the volume is yes, large but no so large that it can sustain wide shots. So this will require some rotoscoping and some other tweaks in the post-production.
Furthermore, if green screens are required at a certain position the makers can pinpoint where they want the greenscreen and bring that on the LED itself. Which basically means that with the help of LED walls, the need for literal green screens gets eliminated.
So will green screens survive this wave of tech innovation?
As answered by Richard Bluff, the VFX Supervisor, ILM — Eventually, Ofcourse we hope never to use green screen. But I still see that there will be a future for it in short term, because there’s likely always a need to remove people, to add additional action behind them. But we are getting to the point where the amount of green screen that’s being used is massively reduced and sky’s the limit right now.
