Online Segmentation of LiDAR Sequences: Dataset and Algorithm

TL;DR

This paper introduces HelixNet, a large LiDAR dataset with detailed labels and timestamps, and Helix4D, a novel efficient transformer-based algorithm for real-time LiDAR sequence segmentation, achieving high accuracy with significantly reduced latency.

Contribution

The paper presents HelixNet, a comprehensive LiDAR dataset, and Helix4D, a new efficient transformer architecture tailored for real-time rotating LiDAR sequence segmentation.

Findings

Helix4D achieves accuracy comparable to state-of-the-art methods.

Helix4D reduces latency by over 5 times and model size by 50 times.

HelixNet enables accurate assessment of real-time segmentation algorithms.

Abstract

Roof-mounted spinning LiDAR sensors are widely used by autonomous vehicles. However, most semantic datasets and algorithms used for LiDAR sequence segmentation operate on $360^\circ$ frames, causing an acquisition latency incompatible with real-time applications. To address this issue, we first introduce HelixNet, a $10$ billion point dataset with fine-grained labels, timestamps, and sensor rotation information necessary to accurately assess the real-time readiness of segmentation algorithms. Second, we propose Helix4D, a compact and efficient spatio-temporal transformer architecture specifically designed for rotating LiDAR sequences. Helix4D operates on acquisition slices corresponding to a fraction of a full sensor rotation, significantly reducing the total latency. Helix4D reaches accuracy on par with the best segmentation algorithms on HelixNet and SemanticKITTI with a reduction of over $5\times$ in terms of latency and $50\times$ in model size. The code and data are available at: https://romainloiseau.fr/helixnet