Effects of Sim2Real Image Translation on Lane Keeping Assist System in CARLA Simulator

TL;DR

This study demonstrates that using DCLGAN for Sim2Real image translation in CARLA improves lane detection accuracy and lane keeping performance in autonomous driving simulations, making virtual testing more realistic and reliable.

Contribution

We developed a real-time DCLGAN-based system for Sim2Real translation in CARLA, optimizing hyperparameters to enhance lane-related visual fidelity and system performance.

Findings

Reduced FID scores indicating more realistic images

Improved lane segmentation accuracy with DCLGAN

Enhanced lane keeping in curved and straight routes

Abstract

Autonomous vehicle simulation has the advantage of testing algorithms in various environment variables and scenarios without wasting time and resources, however, there is a visual gap with the real-world. In this paper, we trained DCLGAN to realistically convert the image of the CARLA simulator and evaluated the effect of the Sim2Real conversion focusing on the LKAS (Lane Keeping Assist System) algorithm. In order to avoid the case where the lane is translated distortedly by DCLGAN, we found the optimal training hyperparameter using FSIM (feature-similarity). After training, we built a system that connected the DCLGAN model with CARLA and AV in real-time. Then, we collected data (e.g. images, GPS) and analyzed them using the following four methods. First, image reality was measured with FID, which we verified quantitatively reflects the lane characteristics. CARLA images that passed through DCLGAN had smaller FID values than the original images. Second, lane segmentation accuracy through ENet-SAD was improved by DCLGAN. Third, in the curved route, the case of using DCLGAN drove closer to the center of the lane and had a high success rate. Lastly, in the straight route, DCLGAN improved lane restoring ability after deviating from the center of the lane as much as in reality.