Scene as Occupancy

TL;DR

This paper introduces OccNet, a vision-centric 3D occupancy reconstruction method that captures detailed scene information, improving various driving tasks like detection, segmentation, and planning, validated on the new OpenOcc benchmark.

Contribution

The paper proposes OccNet, a novel multi-view 3D occupancy reconstruction pipeline with a general occupancy embedding, and introduces OpenOcc, a high-quality occupancy benchmark for autonomous driving.

Findings

Collision rate reduced by 15%-58% in motion planning.

Evident performance gains across multiple driving tasks.

Demonstrates the effectiveness of 3D occupancy representation.

Abstract

Human driver can easily describe the complex traffic scene by visual system. Such an ability of precise perception is essential for driver's planning. To achieve this, a geometry-aware representation that quantizes the physical 3D scene into structured grid map with semantic labels per cell, termed as 3D Occupancy, would be desirable. Compared to the form of bounding box, a key insight behind occupancy is that it could capture the fine-grained details of critical obstacles in the scene, and thereby facilitate subsequent tasks. Prior or concurrent literature mainly concentrate on a single scene completion task, where we might argue that the potential of this occupancy representation might obsess broader impact. In this paper, we propose OccNet, a multi-view vision-centric pipeline with a cascade and temporal voxel decoder to reconstruct 3D occupancy. At the core of OccNet is a general occupancy embedding to represent 3D physical world. Such a descriptor could be applied towards a wide span of driving tasks, including detection, segmentation and planning. To validate the effectiveness of this new representation and our proposed algorithm, we propose OpenOcc, the first dense high-quality 3D occupancy benchmark built on top of nuScenes. Empirical experiments show that there are evident performance gain across multiple tasks, e.g., motion planning could witness a collision rate reduction by 15%-58%, demonstrating the superiority of our method.