Evaluating a GAN for enhancing camera simulation for robotics

TL;DR

This paper evaluates how a GAN can improve simulated images in robotics, reducing the gap between simulation and real-world perception, thereby enhancing robot training and perception accuracy.

Contribution

It provides a quantitative analysis of a GAN's effectiveness in reducing the sim-to-real gap in robotic perception tasks.

Findings

GAN reduces the sim-to-real difference in semantic segmentation.

Enhanced simulation improves object detection performance.

Results demonstrate measurable benefits of GAN-based image enhancement.

Abstract

Given the versatility of generative adversarial networks (GANs), we seek to understand the benefits gained from using an existing GAN to enhance simulated images and reduce the sim-to-real gap. We conduct an analysis in the context of simulating robot performance and image-based perception. Specifically, we quantify the GAN's ability to reduce the sim-to-real difference in image perception in robotics. Using semantic segmentation, we analyze the sim-to-real difference in training and testing, using nominal and enhanced simulation of a city environment. As a secondary application, we consider use of the GAN in enhancing an indoor environment. For this application, object detection is used to analyze the enhancement in training and testing. The results presented quantify the reduction in the sim-to-real gap when using the GAN, and illustrate the benefits of its use.