Major League Baseball is replacing fans using thousands of virtual avatars made using the Unreal Engine.

Major League Baseball is playing games again, but most of the stadiums they’ll be playing in will be empty thanks to COVID-19. And that’s really weird for both the players and fans watching the game at home. It doesn’t sound right. It doesn’t look right, and getting those two things to be the same is crucial for a decent game of baseball.

Some stadiums have taken to replacing missing crowds with cardboard cutouts, but it just ain’t the same. So Fox Sports is doing the next best thing and replacing crowds with virtual ones.

If you watched the games over this past weekend, you might not have even noticed that there’s a pandemic going on. That’s because some neat digital trickery filled in the missing seats with thousands of virtual people thanks to Unreal Engine.

“We had a vision for making our Major League Baseball broadcasts look as natural as they were before COVID," said Brad Zager, executive producer and head of sports operations at Fox Sports, in an interview with Variety. “A lot of that is having a crowd in the stadium.”

So how’d they do it? Fox tapped Silver Spoon Animation to provide essentially the same fake crowds seen in MLB The Show 20 but modified to work during a live broadcast. It uses the Unreal Engine and NVIDIA graphics processing to simulate thousands of individual people in real-time.

“Our pipeline utilizes our expertise in motion capture, animation, and camera tracking, all powered by the latest technology from NVIDIA and Silverdraft,” said Dan Pack, managing director at Silver Spoon. “At its core, our solution is meant to augment a live broadcast, not to be the main attraction. This is even more important for live sports, including baseball, as the virtual fans should add atmosphere but not take away from what matters the most: the game.”

Virtual crowds seen during broadcasts are dynamic and can change during the game. They can perform the wave, of course, but they can also just get up and leave if the home team is getting crushed.

It’s not just virtual crowds, either. Fox is also using sound effects from MLB The Show 20 to make things sound more real too. Those sounds can include cheering, roaring, booing, hissing, and just the regular crowd buzz during a game.

And it’s not just baseball. The NFL, NBA, and other sports leagues are looking to use the same technology and sound effects to replace missing crowds in their games too. Expect this to become the new normal in the coming weeks as Fox Sports irons out the bugs.

Source: Variety, NVIDIA