Researchers at Intel have used machine learning AI to make GTA V look insanely realistic.
“Photorealism has been the defining goal of computer graphics for half a century,” the team write in their paper. “Nevertheless, a look at even the most sophisticated real-time games will quickly reveal that photorealism has not been achieved. An ineffable difference in the appearance of simulation and reality remains.”
They set out to create a system – building on previous attempts using machine learning – that could achieve this kind of photorealism.
During the rendering process, game engines produce intermediate buffers known as “G buffers”. Essentially, this provides detailed data on what will end up on screen, such as geometry, lighting and textures. The main innovation of this algorithm over others is that the team used this intermediate data to train their algorithm and improve the output images.
The team also used photographs collected around Germany by cars with built in cameras (similar to Google Maps) called Cityscapes as training for their neural network to enhance the image output.
Previous attempts at enhancing game graphics as it is rendered have relied on computer labeling of objects. As a result, it’s often unable to be rendered at usual game speeds, or can generate additional unwanted images, such as the trees seen in the skyline demonstrated in the video below.
“Instead of trying to synthesize images, our approach enhances already rendered images, integrates scene information to produce geometrically and semantically consistent images, and does not require any annotation of real data,” the team explain in their paper.
The results of their work are quite astonishing, providing footage that looks pretty close to photorealism, or at least photorealistic enough to make you feel uncomfortable about going on a standard GTA V killing spree.
The shading is better, the hills a lot greener, and roads a lot smoother to look at. At times – though not always – it briefly looks like you are driving around a real city.
The team believe that their system could be improved by integrating it more deeply into game engines, “increasing efficiency and possibly further advancing the level of realism”. More astonishing images produced by their system can be viewed on their GitHub page.