Will AI Make Polygons Obsolete in Video Game Graphics?

GTA IV overlayer with Generative AI

Published: Oct 6, 2024, 2:53PM

An interesting discussion has popped up in the last few days after the release of some text-to-video AI models like Pika 1.5 and Runway Gen-3. The question is whether generative AI is the future of video game graphics. Here are my thoughts on it.

The idea I currently have in mind is to create a low-poly game with low-resolution textures and minimal lighting computation that will be very easy for low-end GPUs to render. Then, those frames could go through generative AI to make them look realistic. The concept is simple, but how feasible is it? We already have Nvidia's DLSS technology, which uses a pretrained AI model to upscale video game frames. Nvidia could potentially train another model specifically designed to achieve what we're trying to accomplish here.

Now, let's say Nvidia developed the technology. There's one significant problem it could create: you will need an Nvidia GPU to run the game. Even if competitors developed the same technology, the game would look quite different depending on the hardware you're using, because the training data of one manufacturer is not the same as that of another. One way to somewhat address this issue would be to provide the GPUs with multiple images of how the game should look. However, this is not a foolproof solution because some parts of the game will still look different.

Moreover, I believe video games exclusive to a console will be immune to this problem, as the game is designed for that hardware, which is the same for every player who has that console. This poses a problem because no one likes console-exclusive games, and it also means that backward compatibility and emulation on different hardware will be an even more monumental task. All of this will, you guessed it, make video game preservation even harder than it already is.

But hey, these are just the thoughts of someone who is very interested in technology and gaming. This field is rapidly evolving, and some challenges will find innovative solutions. So, let me know what you think!