Steering Prediction via a Multi-Sensor System for Autonomous Racing

TL;DR

This paper introduces a novel multi-sensor fusion approach combining 2D LiDAR and event cameras for improved steering prediction in autonomous racing, achieving higher accuracy with lower computational costs.

Contribution

It presents the first multisensor dataset and benchmark for steering prediction, along with a novel, efficient fusion architecture that enhances robustness and accuracy.

Findings

Fusion reduces steering prediction RMSE from 7.72 to 1.28

Proposed method uses only 11% of parameters of second-best method

Our approach outperforms LiDAR-only systems in accuracy and efficiency

Abstract

Autonomous racing has rapidly gained research attention. Traditionally, racing cars rely on 2D LiDAR as their primary visual system. In this work, we explore the integration of an event camera with the existing system to provide enhanced temporal information. Our goal is to fuse the 2D LiDAR data with event data in an end-to-end learning framework for steering prediction, which is crucial for autonomous racing. To the best of our knowledge, this is the first study addressing this challenging research topic. We start by creating a multisensor dataset specifically for steering prediction. Using this dataset, we establish a benchmark by evaluating various SOTA fusion methods. Our observations reveal that existing methods often incur substantial computational costs. To address this, we apply low-rank techniques to propose a novel, efficient, and effective fusion design. We introduce a new fusion learning policy to guide the fusion process, enhancing robustness against misalignment. Our fusion architecture provides better steering prediction than LiDAR alone, significantly reducing the RMSE from 7.72 to 1.28. Compared to the second-best fusion method, our work represents only 11% of the learnable parameters while achieving better accuracy. The source code, dataset, and benchmark will be released to promote future research.