Baseball is back. But the MLB will look a little different in the COVID-19 world, with empty stadiums full of quiet cardboard cutouts instead of thousands of screaming fans to avoid spreading the virus. Unless you're looking at Fox Sports. Starting this weekend, the network will not only bring in fake fan audio data, but also fill the stands with digitally rendered fans for its MLB broadcasts, starting with the Yankees Nationals game on Saturday.
"We believe that the amount and sight of people sitting is part of a program."
All 30 MLB teams will rely on fake fan noise in the actual stadiums (related to Sony's MLB: The Show games), along with the usual announcements in the stadium, walk-up music and other aspects of pomp and circumstances that a accompany major league baseball game. However, Fox Sports is the only broadcaster that has announced plans for virtual fans.
“We believe that the amount and sight of people sitting is part of a program, part of the broadcast of top sports in the major leagues. So we wanted to find a solution, ”said Brad Zager, Executive Producer and Head of Production and Operations at Fox Sports.
"We don't want to fool anyone, it's still about the game. But shot for shot, when you watch a show, you no longer notice that you watch a show with fans in an empty stadium and feel weird – we do want to give people a sense of normalcy, "says Zager." And we felt like going down that path and trying to use a virtual crowd will hopefully make it fit and you can do more focus the game because you don't think of the emptiness of a major league baseball stadium during the season game. "
The effect is a combination of technologies that you may have seen before. The augmented reality software that inserts the crowd is called Pixotope and has worked on AR graphics for things like the Super Bowl and the terrible storm warning demonstrations from The Weather Channel.
It uses graphics (created by the creative agency Silver Spoon Animation) that are integrated into Epics Unreal Engine. Unreal Engine is used here for the same reason that it is popular for creating video games or creating on-set virtual backgrounds for shows like The Mandalorian. Unlike most movie graphics that need to be rendered in post-production, Unreal can be rendered in real time, making it far more suitable for live television.
Finally, SportsMedia Technology (SMT) – the same company that most of the sports-related computer overlays you know like the yellow first-down line in soccer, the screen clock and the scoreboard and much more – takes care of camera tracking Insert graphics into the live camera feeds. Four cameras – high home, centerfield, high first and high third cameras – show the virtual fans.
"We are currently finding out and will continue to develop this process over time."
Fox Sports producers can control how full the virtual "crowds" are for a given game, what weather fans are dressed for, and what percentage of viewers will be home fans compared to away fans, although the company still finds out how some of these decisions are made becomes. "We are currently finding out and will continue to develop this process this weekend and from game to game, week to week," says Zager.
The Fox Sports broadcasts will also incorporate the simulated fan audio played in the stadium into the background of the player's microphone clips (as during the playing fields) and other in-game audios, but the live audio at its own end improve. Also similar to MLS games.
And while the interactions between the virtual crowds and the audio will be fundamental for now (like basic applause, Boos and the Wave), the company hopes to integrate the two better as it gets more practice over the season. "It is something that we hope will be more collaborative and on the same side as we do more and more to combine audio and audience."
Currently, virtual fans are only broadcast on Fox in MLB games, but it is possible that digitally generated fans may appear elsewhere. Zager says the company works with all of its sports sponsors and that "if we approach football and other sports, if we feel that we're going in the right direction, we will start and roll out other sports as well." .