Interactive4D: Interactive 4D LiDAR Segmentation

TL;DR

Interactive4D introduces a novel 4D LiDAR segmentation approach that segments multiple objects across multiple scans simultaneously, improving efficiency and consistency over traditional methods.

Contribution

It is the first model to perform interactive 4D segmentation on superimposed LiDAR scans, leveraging space-time volume for improved accuracy and consistency.

Findings

Achieves state-of-the-art performance on multiple LiDAR datasets

Provides consistent instance IDs over time

Simplifies tracking annotations

Abstract

Interactive segmentation has an important role in facilitating the annotation process of future LiDAR datasets. Existing approaches sequentially segment individual objects at each LiDAR scan, repeating the process throughout the entire sequence, which is redundant and ineffective. In this work, we propose interactive 4D segmentation, a new paradigm that allows segmenting multiple objects on multiple LiDAR scans simultaneously, and Interactive4D, the first interactive 4D segmentation model that segments multiple objects on superimposed consecutive LiDAR scans in a single iteration by utilizing the sequential nature of LiDAR data. While performing interactive segmentation, our model leverages the entire space-time volume, leading to more efficient segmentation. Operating on the 4D volume, it directly provides consistent instance IDs over time and also simplifies tracking annotations. Moreover, we show that click simulations are crucial for successful model training on LiDAR point clouds. To this end, we design a click simulation strategy that is better suited for the characteristics of LiDAR data. To demonstrate its accuracy and effectiveness, we evaluate Interactive4D on multiple LiDAR datasets, where Interactive4D achieves a new state-of-the-art by a large margin. We publicly release the code and models at https://vision.rwth-aachen.de/Interactive4D.