Google’s GameNGen simulates parts of the video game Doom

A team of researchers from Google Research, Google Deep Mind, and Tel Aviv University reports that it is possible to use machine learning applications to recreate and simulate parts or all of an existing video game.

The group published an article on the arXiv preprint server describing their modification of an existing learning application, which they call GameNGen, and used it to reproduce and simulate short sections of the video game Doom.

Efforts to recreate video games with generative AI include two types of work: recreate imagery and recreate action. The process is called “neural rendering,” and it is being studied by several interested groups.

Like other applications of AI, this science relies on the use of diffusion models, generative systems that allow a computer to create new data from old data using special algorithms. In this new study, the research team wanted to determine whether it was possible to faithfully simulate the game Doom by recreating it through machine learning.

The team started with Stable Diffusion 1.4, a diffusion model that Google researchers have been working on for several iterations. Its goal is to create new images using machine learning. After refining the model, the researchers gave it the ability to learn only from video games, instead of everything on the internet, and called it GameNGen.

They trained it with videos from internet sources showing screens of the Doom game in progress while a human was playing. This data was used to teach the new system what the world of Doom is supposed to look like and how the game is supposed to play. They then let it run and found that it could generate new realistic game frames at over 20 frames per second using a single TPU.

The researchers showed clips of Doom created by GameNGen to human reviewers and found that they were unable to tell the difference between the clips and the actual game action more than half the time.

© 2024 Science X Network