<img src = "https://cdn.arstechnica.net/wp-content/uploads/2020/05/nvidia-gamegan-pacman-800×460.png" alt = "To restore Pac-ManYou have to see a lot Pac-Man. "/>
Enlarge /. To rebuild Pac-Man, you need to see a lot of Pac-Man.
Nvidia / Bandai-Namco
You may be familiar with the Infinite Monkey Theorem, an often-quoted (and often misquoted) claim that thousands of monkeys can hit thousands of typewriters and eventually produce an artwork that corresponds to Williams Shakespeare. (Yes, Simpsons did it.)
This week Nvidia confirms that she has taken this theory very seriously with her own twist: an army of AI routines called GameGAN (short for "generative opposing networks") that are trained to create a playable video game from scratch create. More specifically, they have selected one of the largest and most well-known games in the industry and are celebrating their 40th anniversary today: Pac-Man.
If you've seen other computer farms trained on existing games, you've typically learned how to play that game. After watching thousands of hours of a particular game and following the most successful moves and reactions in a verse, these AI routines can control games, repeat and juggle thousands of strategies, and fight people. (Sometimes the results are good for the computers, but not always.)
Nvidia's latest experiment begins in a similar way, as the AI research team trained a farm of four computers – each equipped with a Quadro GV100 GPU for workstations – on 50,000 hours of Pac-Man gameplay. The gameplay was conveniently also played by a separate AI. Based on this footage, the computers in question turned around and created their own clone with identical looks … with a few exceptions.
Look at Pac-Man, write Pac-Man
"Our AI didn't see any code from (Pac-Man), just pixels that came from the game engine," said Nvidia representative Hector Marinez to Ars Technica. "By watching it learned the rules." This included special features such as: Pac-Man's speed, ability to move and inability to walk through walls; the movement patterns of the four spirits; What happens when Pac-Man eats a Kraft pellet? and what happens when ghosts touch Pac-Man, charged or otherwise.
"Each of us could watch people playing Pac-Man for hours, and from there you could potentially be able to write your own Pac-Man game simply by following the rules," said Marinez. "That AI did that."
GameGAN: PAC-MAN Recreated with AI from NVIDIA
Marinez and the rest of the Nvidia researchers available have not clarified exactly which coding subroutine has affected this AI's ability to write its own executable code and whether it relies on existing game engines. (If I could spend hours watching Pac-Man learning how to code video games by hand, I would have a completely different career.) Finally, they gave us a copy of the paper that explains how the Nvidia research team pulled it out. It's complicated. The fastest description consists of a series of three modules (memory, dynamics engine and rendering engine) that are executed as neural networks.
The available researchers also admitted that Nvidia's existing model suffers from loyalty problems. The gameplay below achieves a maximum resolution of approx. 128 × 128 pixels, which is approx. 50 frames per second below the 60 fps standard of the series.
According to Nvidia, this playable version of Pac-Man will be released to the public "this summer", although the company would not clarify whether it will be provided as a downloadable executable or through a limited cloud gaming service such as GeForce Now from Nvidia.
In our interview, Nvidia suggested how this technology could one day revolutionize the work pipeline of a video game studio. The representatives did not do much to support their bold claim that "there is a straight line between Pac-Man and GameGAN to produce games and simulations" that are of the same quality as modern 3D mega-hits. However, the team's researchers were optimistic about AI-driven routines that may support the development of massive virtual worlds as tools to be controlled by human employees.
"We created an AI agent who can only learn the rules of the game through observation," said a representative from Nvidia. "Before you create a tool that adds content to the game, you need to understand the rules." They continued to discuss the wealth of assets (characters, buildings, vehicles) you could expect to find in a massive open-world game – which must first be designed and then encoded to realistically respond to the game world around them. "Think of Grand Theft Auto V. Thousands of artists have worked on producing these worlds for many years. Any tools that help create this content would be very valuable."
Full pretzel loyalty one day?
However, Nvidia made no guess or estimate of how much a computer farm would have to scale to investigate and examine the real-time gameplay of 3D games, let alone the type of video-recorded, real-world environmental studies that the company could use This option to build smarter and more responsive robots. It took 50,000 hours to interpret a single Pac-Man maze, and that in turn was clogged in terms of fidelity. A researcher admitted that higher resolutions "are still an open challenge for this type of network". How many equivalent processing hours does it take to recreate an outdated 3D game like Grand Theft Auto: San Andreas, let alone first-class 3D worlds?
We are not the type to say that it cannot be done. Although we were intrigued by the latest idea, we will keep our excitement until we see a full-resolution version of Q-Bert get its own GameGAN translation. (Really, even Ms. Pac-Man's four-puzzle upgrade would be a good start. Maybe just in time for the 40th anniversary in 2022?)