Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving

TL;DR

This paper introduces LOPR, a self-supervised, stochastic environment prediction framework for autonomous driving that predicts future occupancy grids using generative models, integrating multiple sensor modalities for improved accuracy and consistency.

Contribution

The paper presents LOPR, a novel stochastic occupancy prediction method in latent space that incorporates multiple sensor inputs and offers real-time and refined predictions for autonomous vehicles.

Findings

Outperforms prior methods on nuScenes and Waymo datasets

Provides high-quality real-time predictions with a single-step decoder

Achieves improved temporal consistency and reduced compression losses

Abstract

Environment prediction frameworks are critical for the safe navigation of autonomous vehicles (AVs) in dynamic settings. LiDAR-generated occupancy grid maps (L-OGMs) offer a robust bird's-eye view for the scene representation, enabling self-supervised joint scene predictions while exhibiting resilience to partial observability and perception detection failures. Prior approaches have focused on deterministic L-OGM prediction architectures within the grid cell space. While these methods have seen some success, they frequently produce unrealistic predictions and fail to capture the stochastic nature of the environment. Additionally, they do not effectively integrate additional sensor modalities present in AVs. Our proposed framework, Latent Occupancy Prediction (LOPR), performs stochastic L-OGM prediction in the latent space of a generative architecture and allows for conditioning on RGB cameras, maps, and planned trajectories. We decode predictions using either a single-step decoder, which provides high-quality predictions in real-time, or a diffusion-based batch decoder, which can further refine the decoded frames to address temporal consistency issues and reduce compression losses. Our experiments on the nuScenes and Waymo Open datasets show that all variants of our approach qualitatively and quantitatively outperform prior approaches.