An AI built Pac-Man from the ground up just by watching people play Pac-Man

Pac-Man, one of gaming's most recognizable and influential classics, is turning 40 this year. With such a legacy, the Bandai Namco game solidifies its place as one of the arcade's longest-lasting offerings — a game so familiar and accessible that anyone could figure out how to play. Or, if you happen to be an advanced piece of artificial intelligence, figure out how to program the game.

That's what happened when Nvidia Research gave its GameGAN (which relies on Generative Adversarial Networks) 50,000 videos of people racing through the various mazes and chomping the various dots/pellets of Pac-Man. Just from watching these videos of people playing the arcade classic, the GameGAN figured out how the game works, well enough to make its own version.

If you think deepfakes are scary, just try watching an AI reverse-engineer software simply by observing it:

That's some hacking the Matrix business right there.

Writing on the Nvidia blog, Isha Salian explains that the GameGAN's version of the game — made without the AI ever seeing a line of code — is possible because, using "two competing neural networks, a generator and a discriminator, GAN-based models learn to create new content that’s convincing enough to pass for the original."

“This is the first research to emulate a game engine using GAN-based neural networks,” said NVIDIA researcher and project lead Seung-Wook Kim. “We wanted to see whether the AI could learn the rules of an environment just by looking at the screenplay of an agent moving through the game. And it did.”

The AI learned the rules, created the maze and characters (blurry as they may be, which is par for the course of AI-generated images), and made a human-playable version of the game using images and keystroke information.

Now the floodgates are open to how the gaming world — and beyond — will be able to use this technology. Making a video game that dominated the '80s is one thing, but these GANs could function as simulators for all sorts of gaming environments ... as long as they could watch enough work in those virtual spaces to understand how they work. It might not happen anytime soon, but the top games every year may eventually consist of many key AI-created elements.

GameGAN was created by Kim, NVIDIA’s Toronto research lab director Sanja Fidler, researcher Jonah Philion, University of Toronto student Yuhao Zhou, and MIT professor Antonio Torralba. Nvidia plans to release its AI-developed Pac-Man later this year.