You know the blue screen (and green screen). You know it becomes the mist-covered planet, deserted jungle, packed athletic stadium, background for a bar, and literally any other place or objects you want it to be. What happens when the imaginary world is already there? The moment you step inside the volume, you are just there. “The Volume” is the awesome-sounding name for the union of high resolution LED panels – imagine breath-taking huge TVs set up all around a stage – the physical set design matching the panels and 3D models toppled into an environment exactly as they do in an interactive media like a puzzle video game. Then it responses to camera movements to simulate the real world. Disney+’s The Mandalorian used this technique.
In an interview with Vox, Lead Compositor at Industrial Light & Magic Charmaine Chan revealed how they achieved the VFX without green screens. She said that the use of “The Volume” could get really confusing on the set. When shooting a scene all day in the same exact scene and you are at that location, it doesn’t feel like it is something fake. It just feels like the appendage of a regular stage. You have got to be more careful because there are times when people don’t see where the stage ends and where the LEDs are placed.
How to Use a Green Screen
How green screen is being replaced exactly is easily seen in this latest Disney+ project. Charmaine is credited as part of the Brain Bar – the team of visual effects artists that worked on this system. One might make adjustments to models, like a rock or spaceship in panels, while another might tweak live animations, like burning fire.
Charmain often adjusted color, and it was a delightful experience, according to her. She described it as those telephone fundraising on PBS because it just like those rows of people ready with phones. But instead of phones, Charmain and her Brain crew members had computers and walkie talkies.
Mandalorian VFX supervisor Ian Milham tweeted that the set crew and Brain Bar operating the panels was a tremendous help to them in radically changing environments in minimal time or be on set as they launched it into space. Charmaine further added that her normal working life is very much behind the computer in a dark room, somewhere in the corner.
Now she is actually in there with the gaffer, with the prop designers, with the set designers, most people that she would never see otherwise because she is in the post-production process. She found this experience to be very exhilarating. This is how green screen is being replaced.
But sets like this one were not just fun for Charmaine. They helped remove creative roadblocks. As a compositor, she and her team are the ones who take all the CG elements, take all the renders, and put them together to make it look like it is a seamless, integrated photo. So think of it as an advanced Photoshop, but they are dealing with moving imagery.
Charmaine worked on the scene in The Last Jedi when Kylo is dueling in front of a green screen. On your lucky day, this screen will be evenly lit, with no seams. That is never the scenario, however. The VFX team has to spend time, almost frame by frame, making sure they can remove that green screen so that they could put Kylo on top of that. Removing a green screen is still a tedious job to do even today.
For one, it doesn’t work with characters that are green i.e., Yoda. The perspective of the background also doesn’t change as it does in a natural setting, which has to be designed into the last final composite. Ditching the green screen and projecting or playing the image behind the actors gets you closer, but not quite there. You can get an illusion of depth and better light with details of the environment. Instead of the green screen going all over the actor, red desert and you get blue sky actually lighting them up to get a natural look.
This is a basic technique that has worked in many movies from 2001 to Oblivion, but the proper perspective shift is missed in it, or parallax, in the background, since it is just a video playing on a TV. “The Volume” solves some of those problems. You can also adjust light and objects that are on the fly.
And the reflections work perfectly because other screens are reflected over the objects instead of the green screen – which was critically important for Mando, who is the show’s main character. His whole armor was reflective from head to toe; whether it be his pauldron or his helmet, you just can’t avoid seeing things being reflected. It was so creating volume where they could literally close up the whole thing into one giant circle and have an environment all across these screens. Now imagine how green screen Mando would have looked so dull on the surfaces.
The Brain bar could focus on details that made the final product as seamless as possible, which was still a lot of work. Color correction was very important when they were dealing with the bigger parts of the set. So whether that is the dirt on the ground versus dirt in the digital scene, they had to color correct it. It is also how green screen can affect color correctness but it takes less effort.
We can imagine many creative breakthroughs that this makes possible for VFX artists job. The compositors will no longer have to wonder how green screen keying or extraction will take so much time, ever again. The VFX team can now help finalize a shot in camera. It just makes it a more creative and cohesive filmmaking process, and this puts us right in there next to everyone else who is creating these shows or films.