Case Study: Ross Video drives the future of immersive broadcast with virtual set extensions

Virtual production is now a broadcast requirement for merging the physical and digital worlds seamlessly into one cohesive viewing experience. From broadcast and cable networks to streaming services and online productions, more content producers are deploying virtual set extensions, especially for high-profile programming like sports or election coverage, with many designed and managed by Ross Video’s Rocket Surgery Creative & Professional Services team leveraging the company’s Voyager and Lucid Studio virtual production solutions.

“Live broadcast coverage has turned into a competitive race to continually improve and keep giving viewers that ‘you are there’ feeling,” said Jim Doyle, director of creative services at Rocket Surgery. “With our virtual set extension technology, broadcasters can keep their graphics and data presentation dynamic with the creative flexibility to make changes and updates on the fly and give viewers an engaging, immersive experience.”

View on Giphy

For recent election night coverage by a cable network, the Rocket Surgery team created a trackless, augmented reality (AR) environment that extended the physical set into an immersive virtual world. The team created this environment built on multiple discrete post-process volumes to enable seamless graphic transitions from exteriors to interiors, including day and night lighting scenarios.

Powered by Ross Video’s Voyager, built on Epic Game’s Unreal Engine, the massive, highly detailed set added the ability to cull offscreen objects even when they had a custom depth stencil value, which wasn’t previously possible.

Data integration was a centerpiece of this project to drive the numbers and text displayed on-screen, as well as determine how and when the virtual world would animate. Ross Video’s Datalinq for Voyager enabled Rocket Surgery to easily bind real- time elections data to 3D infographics rendered in Voyager.

Rocket Surgery created a blueprint for evaluating the network’s data to quickly identify any changes, download candidate photos, and display race updates almost instantly.

This set extension project used a dynamic material instance workflow to create DMI files, download images from a NAS, replace texture parameters and apply the new DMI to specified meshes.

Advertisement

This project, like many other recent virtual set projects, showcased the latest version of the Ross Dashboard control system and Rosstalk GPI workflow which uses a plain text-based protocol to control the switcher and virtual set from an external device.

All functions of the virtual set that weren’t automated from network data were controlled from Dashboard using GPI commands. Custom Dashboard functions designed by Rocket Surgery enable operators to quickly change which graphics engines receive specific commands. For example, this is helpful when production wants one engine prepped for an AR set extension shot and another to be ready with a different shot in the virtual world.

“On this most recent project, we had a lot of unique sequences and actors to be controlled, so we leveraged the capabilities of RossTalk GPI commands to trigger a variety of events,” said Terry Daily, virtual creative services manager at Rocket Surgery. “The flexibility to choose different methods to control and trigger the project is a big advantage to the Lucid and Voyager workflow.”

GPI commands were used to play level sequences, reposition the virtual set, update poll closing times, load and unload levels, and blend between cameras.

The operator’s custom Dashboard was constantly changing, and it was helpful to be able to change the commands and the layout all at once.