Semantic Road Layout Understanding by Generative Adversarial Inpainting

TL;DR

This paper introduces a GAN-based inpainting approach to remove dynamic objects from road scenes, enabling better understanding of static environment components for autonomous driving.

Contribution

It presents a novel semantic segmentation inpainting model using GANs to focus on static scene understanding, evaluated on Cityscapes and a new synthetic dataset.

Findings

Effective removal of dynamic objects in road scenes

Improved static scene segmentation accuracy

Comparable or superior to baseline methods in experiments

Abstract

Autonomous driving is becoming a reality, yet vehicles still need to rely on complex sensor fusion to understand the scene they act in. The ability to discern static environment and dynamic entities provides a comprehension of the road layout that poses constraints to the reasoning process about moving objects. We pursue this through a GAN-based semantic segmentation inpainting model to remove all dynamic objects from the scene and focus on understanding its static components such as streets, sidewalks and buildings. We evaluate this task on the Cityscapes dataset and on a novel synthetically generated dataset obtained with the CARLA simulator and specifically designed to quantitatively evaluate semantic segmentation inpaintings. We compare our methods with a variety of baselines working both in the RGB and segmentation domains.