LiDAR View Synthesis for Robust Vehicle Navigation Without Expert Labels

TL;DR

This paper introduces a method to synthesize additional LiDAR point clouds from novel viewpoints using mesh reconstruction and ray casting, enhancing vehicle navigation robustness without requiring expert labels.

Contribution

It presents a novel LiDAR view synthesis technique that generates training data for self-driving models without needing expert driving labels, improving robustness.

Findings

Synthesized LiDAR views improve model robustness.

The approach outperforms existing data augmentation methods.

Effective in diverse driving scenarios.

Abstract

Deep learning models for self-driving cars require a diverse training dataset to manage critical driving scenarios on public roads safely. This includes having data from divergent trajectories, such as the oncoming traffic lane or sidewalks. Such data would be too dangerous to collect in the real world. Data augmentation approaches have been proposed to tackle this issue using RGB images. However, solutions based on LiDAR sensors are scarce. Therefore, we propose synthesizing additional LiDAR point clouds from novel viewpoints without physically driving at dangerous positions. The LiDAR view synthesis is done using mesh reconstruction and ray casting. We train a deep learning model, which takes a LiDAR scan as input and predicts the future trajectory as output. A waypoint controller is then applied to this predicted trajectory to determine the throttle and steering labels of the ego-vehicle. Our method neither requires expert driving labels for the original nor the synthesized LiDAR sequence. Instead, we infer labels from LiDAR odometry. We demonstrate the effectiveness of our approach in a comprehensive online evaluation and with a comparison to concurrent work. Our results show the importance of synthesizing additional LiDAR point clouds, particularly in terms of model robustness. Project page: https://jonathsch.github.io/lidar-synthesis/