Virtual production is - in the filmmaking world, at least - a rather new method that has been put to use in many big productions. You could say that the “breakout” of the practice came due to its usage in Disney’s 2019 movie The Lion King, directed by Jon Favreau.
Ok, I still didn’t answer the title’s question. What exactly is this process? So, virtual production essentially consists of establishing major visual elements of the film before going to the set and also being able to make minor changes of such elements on the spot. Those visuals can include anything from background scenarios and set dimensions (and depth) to asset placement and exact camera movements.
So, the movie or TV series that eventually uses this technology would technically have some kind of virtual cinematography? Yes, it would.
I’m aware that this explanation may sound a bit confusing, especially for less learned people in the filmmaking area. However, I guarantee you that is a far simpler concept to understand than it seems.
After proven itself as a very efficient filmmaking method in The Lion King, the virtual film production was used massively in a little show called The Mandalorian (also a Jon Favreau project). And to better illustrate how this type of highly advanced filmmaking technique actually works, here’s a video explaining how it was used in the Disney’s hit TV series.
It can seem like a very complex technology from another world. One thing is to talk about green screens, a method that allows a film set to be prepared for VFX work that will happen entirely in the post production. But with virtual production, the VFX is already part of the studio and it’s happening right in front people’s eyes, with contrast, colors, and lighting changes being effortlessly made to the whole scenario.
And what some people would like to know is: How the hell does this tech work?
The basis for the method to work in a show like The Mandalorian is the collaboration with Unreal Engine, which is a game engine platform with a major variety of software tools that - besides being key to game developing - help emulate live action film scenarios in a VR space with LED panels.
So, what I’m telling you is that the film industry is essentially using gaming technology to improve their industry, save money and revolutionize the way VFX is done for movies? Yes, that’s exactly what I’m telling you.
Other platforms, such as the Technicolor Virtual Production pipeline, a data tracking and VFX authoring platform with multi-use and collaborative purposes, is also used during the virtual production process. Through it, people can track concept arts, camera movements, lenses to assets, etc. It’s basically a giant database with various visual concepts that filmmakers can choose from and make changes through it.
Important to remember: most of the virtual production work is done BEFORE filming actually starts. However, small changes and other features can be applied to the finished work during filming, if necessary.
Virtual production is definitely not one of the most simple (or cheap) filmmaking techniques available right now, but is certainly where the future of movies is heading. That’s because the method has cemented itself as a very advantageous option.
The main benefits of opting for virtual production include:
● Less time and money consuming
By discarding the possibility of filming on location, things can get a lot simpler. You don’t have to scout the locations, you don’t pay for transportation of any crew members or filming equipment, and you don’t have to deal with bad weather delaying the schedule. By eliminating all of that, you can save a whole bunch of money, and put the extra time you managed to save into other important activities, like composing shots and directing actors.
Not to mention, you can considerably reduce the possibility of having to do reshoots or post-production adjustments of various kinds.
● More creative planning
Virtual production allows you to get much more time in the actual location of filming. That means you have more time to work on shot composition, actors’ blocking, their movements, etc. Space for any type of sloppiness is immensely diminished.
And if you’re not liking something about the scenario, lighting or assets of a scene? No worries. The flexibility of this kind of production allows you to make changes or experimentations instantly, with not a single thing having to be altered practically and with no extra money being spent.
● Full control of the environment and lighting
With the amount of control that virtual production and the game engine puts in your hands, you can do things like move buildings, trees and mountains, completely reshape landscapes, and even administer lighting to, for example, keep a twilight going for the whole day.
The lighting possibilities of virtual production allow you to achieve great results already on the set, without having to worry about future post-production adjustments that a green screen shoot usually demands.
You can also transit from one scene to the next, switching between completely different landscapes, in just one command.
● Real-time global collaboration
Filmmakers can digitally interact with objects, characters and fully formed scenarios created by other artists. They can test all of these assets, placing them on their already finished scenarios and seeing how they can fit the scene.
Some of the main positions available for those interested in working with virtual film production are:
● Motion capture supervisor
This position works directly with the motion capture data that comes from performers' suits and camera tracking systems. Motion capture allows performers to give life to digitally created characters through their own movements and expressions.
The supervisor has to make sure the capturing is happening as it should. They have to check the actors’ moves, as well as having a deep understanding of real-world physics and camera movements.
Average Salary (per year): U$89,800.
● Real-time compositor
They’re the ones responsible for producing any 2D or 3D visual elements needed for the production during filming. This kind of VFX compositing consists of the creation of several separate layers that have to remain individually adjustable.
Often, real-time composers only have to specialize themselves in only one of the platforms that contain the software packages necessary for them to work, like Unreal Engine, DaVinci Resolve or Nuke.
Average Salary (per year): U$165,000
● LED engineer
The LED panels, on a film set working through virtual production, are simply the basis of the whole technique. It’s in those panels that the visual conceptualization for each scene is showcased. And the LED engineers are the ones responsible for making sure that both the LED panels and video processors are operating efficiently.
Average Salary (per year): U$90,153
● Systems administrator
This is a complex position that requires a very broadly skilled person, who will oversee many areas included on a virtual production-based film set, such as: media structure, spectrum management and network infrastructure.
The person willing to work as a systems administrator has to be an IT professional. Due the complex nature of the job, it’s very common for productions to hire more than one person for the position, with each one of them being directed to the IT area they’re most acquainted with (asset management, video I/O troubleshooting, network support, etc).
Average Salary (per year): U$83,056
Other jobs
● Systems technical director: U$126,119
● Video engineer: U$67,000
● Virtual camera operator: U$65,000
● Virtual production supervisor: U$200,000
For many of the job positions of the virtual film production world, a degree in electrical engineering, IT , graphic design or, obviously, filmmaking areas would be a great start. However, further specialization courses teaching about the specific equipment and softwares utilized in the area would most likely be needed.
This specialization course offered by NFTS runs for twenty four weeks on a combination of in-person practical learning and online remote learning, with lessons taking place during weekday evenings and weekend days.
The course’s purpose is to teach students the core skills and filmmaking techniques required to understand the VP industry and be successful in it. Some activities included in the course that give students pivotal in-depth knowledge about VP are:
● Advance learning on Unreal Engine and its tools.
● Walkthrough of the LED Wall setups to understand their main functionalities and obtain basic knowledge on how they operate.
● How to merge digital and physical props in a natural way.
● Design workflows.
● Learning about responsibilities of the main roles within a virtual production, such as a Director of Photography, Production Designer, Technical Artist, and more.
To take the course NFTS requires students to have at least 18 years old and two years of industry experience, and preferably have a background in areas such as VFX, 3D animation, graphic design or as a member of a technical film crew. Students also must have free access to Unreal Engine on their PCs or laptops.
The course will leave students ready for any entry-level job on a virtual film production set, such as an assistant, trainee and basically any type of junior position. Being able to acquire some practical experience on those sets, NFTS alumni will be able to safely choose a VP specialty and deepen their knowledge in a specific area, climbing up the industry ladder.
The virtual film production industry is definitely a working area going through a big expansion. It’s getting big and getting big fast, moving a lot of money around, and with more people getting interested as an occupation or investment each day.
Also, most jobs you can find in VP have really attractive pay rates. People that have an interest not only in film, but in the other areas we mentioned here, and manage to get some work in the area, are no doubt entering an industry that is already really nicely valued, and has everything to get even more.