Vision-based 3D occupancy prediction in autonomous driving: a review and outlook

TL;DR

This paper reviews recent advances in vision-based 3D occupancy prediction for autonomous driving, highlighting challenges, categorizing methods, and outlining future research directions to improve perception systems.

Contribution

It provides the first comprehensive survey of vision-based 3D occupancy prediction, analyzing methods across feature enhancement, deployment, and label efficiency, and discusses future outlooks.

Findings

Survey of recent methods and trends

Analysis of challenges and potentials

Identification of future research directions

Abstract

In recent years, autonomous driving has garnered escalating attention for its potential to relieve drivers' burdens and improve driving safety. Vision-based 3D occupancy prediction, which predicts the spatial occupancy status and semantics of 3D voxel grids around the autonomous vehicle from image inputs, is an emerging perception task suitable for cost-effective perception system of autonomous driving. Although numerous studies have demonstrated the greater advantages of 3D occupancy prediction over object-centric perception tasks, there is still a lack of a dedicated review focusing on this rapidly developing field. In this paper, we first introduce the background of vision-based 3D occupancy prediction and discuss the challenges in this task. Secondly, we conduct a comprehensive survey of the progress in vision-based 3D occupancy prediction from three aspects: feature enhancement, deployment friendliness and label efficiency, and provide an in-depth analysis of the potentials and challenges of each category of methods. Finally, we present a summary of prevailing research trends and propose some inspiring future outlooks. To provide a valuable reference for researchers, a regularly updated collection of related papers, datasets, and codes is organized at https://github.com/zya3d/Awesome-3D-Occupancy-Prediction.