NVIDIA’s AI App Can Transform Simple Doodles into Spectacular Landscapes

GauGAN - NVIDIA Landscapes Using AI

Researchers at NVIDIA have already developed AI technology that can generate photos of fake people, and now they're taking things a step further. Continuing their exploration of what's possible using generative adversarial networks, or GANs, they created a program that can turn a simple doodle into a beautiful landscape “photograph.”

With a few clicks and drags in an app that looks like MS Paint, anyone can create a scheme that is then translated into a startlingly realistic landscape. NVIDIA's deep learning model takes the basic sketch and uses an algorithm to create a photorealistic masterpiece with ease.

Named GauGAN—a nod to famed post-Impressionist painter Paul Gauguin—the interactive app used to produce the fake landscapes is easy to manipulate. Different colors are assigned to elements like sky, mountain, rock, plant, and road. It's even possible to select weather elements like snow, and the program is smart enough to distinguish between details like gravel and sand.

“It’s like a coloring book picture that describes where a tree is, where the sun is, where the sky is,” explains Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA. “And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colors, based on what it has learned about real images.”

GauGAN - NVIDIA - AI Landscape Maker

This means that if you place a body of water next to trees, the program will know to reflect the trees in the pond in the resulting photograph. You can even show the same landscape in different seasons by toggling between grass and snow. NVIDIA hopes that the program will be useful for people who design virtual worlds, such as architects and landscape designers.

As the program was trained using millions of images, it has a rich knowledge that helps it fine-tune its landscapes. “This technology is not just stitching together pieces of other images, or cutting and pasting textures,” Catanzaro said. “It’s actually synthesizing new images, very similar to how an artist would draw something.”

Interestingly, the underlying technology used in GauGAN can also fill in features like buildings and people, which will surely be a further development.In June, a research paper on GauGAN will be presented at the CVPR conference. In the meantime, attendees of NVIDIA's GPU Technology Conference in March 2019 had a chance to test out these artificial landscapes.

GauGAN is incredible AI technology that turns simple doodles into photorealistic landscapes.

GauGAN - NVIDIA - AI Landscape Maker

GauGAN - NVIDIA - AI Landscape Maker

The interactive app is so sophisticated that it knows to adjust the lighting of the entire landscape depending on the time of day.

GauGAN - NVIDIA - AI Landscape Maker

GauGAN - NVIDIA - AI Landscape Maker

Learn more about the incredible deep learning model created by NVIDIA's researchers.

h/t: [PetaPixel]

All images via NVIDIA/YouTube.

Related Articles:

This Website Uses AI Technology to Generate Faces That Aren’t Real

Interview: Before and After Photos of Teens Asked to Edit Their Selfies for Social Media

This Japanese Fashion Model Isn’t a Human, She’s Computer Generated

Ingenious Weather Poster Uses “Smart Ink” to Give Real-Time Weather Updates

Jessica Stewart

Jessica Stewart is a Staff Editor and Digital Media Specialist for My Modern Met, as well as a curator and art historian. Since 2020, she is also one of the co-hosts of the My Modern Met Top Artist Podcast. She earned her MA in Renaissance Studies from University College London and now lives in Rome, Italy. She cultivated expertise in street art which led to the purchase of her photographic archive by the Treccani Italian Encyclopedia in 2014. When she’s not spending time with her three dogs, she also manages the studio of a successful street artist. In 2013, she authored the book 'Street Art Stories Roma' and most recently contributed to 'Crossroads: A Glimpse Into the Life of Alice Pasquini'. You can follow her adventures online at @romephotoblog.
Become a
My Modern Met Member
As a member, you'll join us in our effort to support the arts.
Become a Member
Explore member benefits

Sponsored Content