A Sim-to-Real Vision-based Lane Keeping System for a 1:10-scale Autonomous Vehicle

TL;DR

This paper presents a vision-based lane keeping system for a 1:10-scale autonomous vehicle that effectively transfers from simulation to real-world operation using CNNs and tailored control strategies.

Contribution

It introduces a novel Sim2Real approach for vision-based lane keeping on a scaled vehicle, including a compact CNN training method and a tailored control system.

Findings

Successful real-time operation on low-level hardware

Effective simulation-to-reality transfer demonstrated

Validated in controlled laboratory setting

Abstract

In recent years, several competitions have highlighted the need to investigate vision-based solutions to address scenarios with functional insufficiencies in perception, world modeling and localization. This article presents the Vision-based Lane Keeping System (VbLKS) developed by the DEI-Unipd Team within the context of the Bosch Future Mobility Challenge 2022. The main contribution lies in a Simulation-to-Reality (Sim2Real) GPS-denied VbLKS for a 1:10-scale autonomous vehicle. In this VbLKS, the input to a tailored Pure Pursuit (PP) based control strategy, namely the Lookahead Heading Error (LHE), is estimated at a constant lookahead distance employing a Convolutional Neural Network (CNN). A training strategy for a compact CNN is proposed, emphasizing data generation and augmentation on simulated camera images from a 3D Gazebo simulator, and enabling real-time operation on low-level hardware. A tailored PP-based lateral controller equipped with a derivative action and a PP-based velocity reference generation are implemented. Tuning ranges are established through a systematic time-delay stability analysis. Validation in a representative controlled laboratory setting is provided.