UnO: Unsupervised Occupancy Fields for Perception and Forecasting

TL;DR

This paper introduces UnO, an unsupervised 4D occupancy field model for perception and forecasting in self-driving, which outperforms supervised methods especially with limited labeled data and improves object recall in spatio-temporal predictions.

Contribution

UnO presents a novel unsupervised approach to learn continuous 4D occupancy fields from LiDAR data, enabling better transferability and performance in perception and forecasting tasks.

Findings

State-of-the-art performance on Argoverse 2, nuScenes, and KITTI datasets.

Outperforms supervised methods in BEV semantic occupancy forecasting with limited labels.

Achieves higher recall of relevant objects in spatio-temporal predictions.

Abstract

Perceiving the world and forecasting its future state is a critical task for self-driving. Supervised approaches leverage annotated object labels to learn a model of the world -- traditionally with object detections and trajectory predictions, or temporal bird's-eye-view (BEV) occupancy fields. However, these annotations are expensive and typically limited to a set of predefined categories that do not cover everything we might encounter on the road. Instead, we learn to perceive and forecast a continuous 4D (spatio-temporal) occupancy field with self-supervision from LiDAR data. This unsupervised world model can be easily and effectively transferred to downstream tasks. We tackle point cloud forecasting by adding a lightweight learned renderer and achieve state-of-the-art performance in Argoverse 2, nuScenes, and KITTI. To further showcase its transferability, we fine-tune our model for BEV semantic occupancy forecasting and show that it outperforms the fully supervised state-of-the-art, especially when labeled data is scarce. Finally, when compared to prior state-of-the-art on spatio-temporal geometric occupancy prediction, our 4D world model achieves a much higher recall of objects from classes relevant to self-driving.