Procedural 3D Terrain Generation using Generative Adversarial Networks

TL;DR

This paper presents a novel method using GANs to generate realistic 3D terrains by synthesizing satellite images and corresponding height maps, enabling the creation of diverse and plausible landscapes for open world games.

Contribution

It introduces a combined GAN and CGAN framework for procedural 3D terrain generation from satellite imagery, a novel approach in game environment creation.

Findings

Generated satellite images closely resemble real landscapes

Produced plausible height maps from synthetic images

Enabled realistic 3D terrain construction

Abstract

Procedural 3D Terrain generation has become a necessity in open world games, as it can provide unlimited content, through a functionally infinite number of different areas, for players to explore. In our approach, we use Generative Adversarial Networks (GAN) to yield realistic 3D environments based on the distribution of remotely sensed images of landscapes, captured by satellites or drones. Our task consists of synthesizing a random but plausible RGB satellite image and generating a corresponding Height Map in the form of a 3D point cloud that will serve as an appropriate mesh of the landscape. For the first step, we utilize a GAN trained with satellite images that manages to learn the distribution of the dataset, creating novel satellite images. For the second part, we need a one-to-one mapping from RGB images to Digital Elevation Models (DEM). We deploy a Conditional Generative Adversarial network (CGAN), which is the state-of-the-art approach to image-to-image translation, to generate a plausible height map for every randomly generated image of the first model. Combining the generated DEM and RGB image, we are able to construct 3D scenery consisting of a plausible height distribution and colorization, in relation to the remotely sensed landscapes provided during training.