The company Nvidia has developed a "hybrid graphic system" based on artificial intelligence, able to draw urban landscapes in real time. Trained in a video shot in the real world, the new model of artificial intelligence has learned to generate new virtual spaces from scratch.
For the design of buildings, cars, pedestrians and other objects responsible generative-adversary of the network (GAN). This system "bleeds" between them two neural networks: the one that directly creates the image ("generator"), and that analyzes it (the "discriminator"), comparing the resulting work with the reference images.
The graphics are generated in several phases, writes The Verge. In the beginning, researchers "fed" neural networks with publicly available video data, commonly used to train autonomous driving algorithms. Each frame has been assigned to a specific category: sky, trees, road, buildings and so on. Then came the GAN, which created new versions of these objects, learning from video footage from the real world.
Then, using the Unreal Engine 4 game engine (used by Fortnite, PUBG, Gears of War 4 and other achievements), Nvidia's engineers created the virtual topology of the virtual environment. Using UE4 as a framework, AI algorithms were able to generate graphs for each object, linking them to the game engine models.
To demonstrate the work of technology, the researchers created a simple driving simulator in which players can drive in residential areas and admire the architecture. In addition to video games, the "hybrid graphic system" of Nvidia can be used in film and virtual reality, used as a learning environment for robots and self-driving cars.