All the sessions from Transform 2021 are available on-demand now. Watch now.
Nvidia Research today unveiled GauGAN, a generative adversarial AI system that lets you create lifelike landscape images that never existed.
The research and a demo of the new system were shown off today at the GPU Tech Conference (GTC) in San Jose, California.
GauGAN builds upon learnings from the Pix2Pix system introduced last year that can render virtual worlds, said Nvidia VP of applied deep learning research Bryan Catanzaro, but Pix2Pix can’t paint landscapes because doing so leaves artifacts in the resulting image.
GauGAN’s neural network is trained with one million open source Flickr images and imbued with an understanding of the relationship between more than 180 objects like snow, trees, water, flowers, bushes, hills, or mountains.
That understanding of how objects relate to each other means a tree next to water will show a reflection, or when the season changes and there’s snow on the ground, trees will be depicted without leaves.
Style transfer is also possible so an image can adopt a warm sunset glow or display the cooler lights of a city skyline.
The GauGAN app uses a segmentation map, which acts sort of like a coloring book that describes where objects are but provides no detail, as seen below.
A paper by Nvidia principal research scientist Ming-Yu Liu and others that details the creation of GauGAN and its spatially adaptive denormalization method for photo manipulation was released today. The paper was also accepted for oral presentation at the Computer Vision and Pattern Recognition (CVPR) 2019 conference to be held in June in Long Beach, California.
Also making its debut today is the Nvidia AI Playground, a website where people can tinker with a variety of trained neural nets like GauGAN that use powerful AI to distort visuals or create lifelike images.
GauGAN is the latest reality-bending AI system from Nvidia, creator of deepfake tech like StyleGAN, which can generate lifelike images of people who never existed, and Nvidia open sourced last month.
Responding to a question of Nvidia releasing another system that can make people question what’s real, Catanzaro said that as a researcher, “this has been my dream” because the tech necessary to achieve such results represents progress — but on the other hand, it is tech that some people may misuse.
“Personally, I believe it’s a trust issue and not a tech issue,” Catanzaro said.
Catanzaro said he’s most excited about GauGAN being adopted by video game designers creating the landscape of virtual worlds and envisions applications of the technology for creatives who use storyboards to lay out their creations.
VentureBeatVentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:
- up-to-date information on the subjects of interest to you
- our newsletters
- gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
- networking features, and more