UniLiPs: Unified LiDAR Pseudo-Labeling with Geometry-Grounded Dynamic Scene Decomposition

TL;DR

This paper presents an unsupervised, multi-modal pseudo-labeling approach for LiDAR data that leverages geometric and temporal consistency to generate dense 3D semantic labels and detect moving objects without manual annotations.

Contribution

It introduces a novel iterative update rule that enforces joint geometric-semantic consistency and utilizes foundation models to improve 3D labeling and object detection in LiDAR data.

Findings

Outperforms existing pseudo-labeling methods in semantic segmentation and detection.

Improves depth prediction accuracy by over 50% in distant ranges.

Produces robust generalization across multiple datasets.

Abstract

Unlabeled LiDAR logs, in autonomous driving applications, are inherently a gold mine of dense 3D geometry hiding in plain sight - yet they are almost useless without human labels, highlighting a dominant cost barrier for autonomous-perception research. In this work we tackle this bottleneck by leveraging temporal-geometric consistency across LiDAR sweeps to lift and fuse cues from text and 2D vision foundation models directly into 3D, without any manual input. We introduce an unsupervised multi-modal pseudo-labeling method relying on strong geometric priors learned from temporally accumulated LiDAR maps, alongside with a novel iterative update rule that enforces joint geometric-semantic consistency, and vice-versa detecting moving objects from inconsistencies. Our method simultaneously produces 3D semantic labels, 3D bounding boxes, and dense LiDAR scans, demonstrating robust generalization across three datasets. We experimentally validate that our method compares favorably to existing semantic segmentation and object detection pseudo-labeling methods, which often require additional manual supervision. We confirm that even a small fraction of our geometrically consistent, densified LiDAR improves depth prediction by 51.5% and 22.0% MAE in the 80-150 and 150-250 meters range, respectively.