EFFOcc: Learning Efficient Occupancy Networks from Minimal Labels for Autonomous Driving

TL;DR

This paper introduces EFFOcc, an efficient framework for 3D occupancy prediction in autonomous driving that reduces model complexity and label requirements while maintaining state-of-the-art accuracy across multiple benchmarks.

Contribution

The paper presents a fusion-based OccNet using simple 2D operators and a multi-stage distillation method to achieve high accuracy with minimal labels and low computational cost.

Findings

Achieves 51.49 mIoU with 21.35M parameters on Occ3D-nuScenes.

Attains 94.38% of full-label performance with only 40% labeled data.

Outperforms existing methods in accuracy and efficiency on multiple benchmarks.

Abstract

3D occupancy prediction (3DOcc) is a rapidly rising and challenging perception task in the field of autonomous driving. Existing 3D occupancy networks (OccNets) are both computationally heavy and label-hungry. In terms of model complexity, OccNets are commonly composed of heavy Conv3D modules or transformers at the voxel level. Moreover, OccNets are supervised with expensive large-scale dense voxel labels. Model and data inefficiencies, caused by excessive network parameters and label annotation requirements, severely hinder the onboard deployment of OccNets. This paper proposes an EFFicient Occupancy learning framework, EFFOcc, that targets minimal network complexity and label requirements while achieving state-of-the-art accuracy. We first propose an efficient fusion-based OccNet that only uses simple 2D operators and improves accuracy to the state-of-the-art on three large-scale benchmarks: Occ3D-nuScenes, Occ3D-Waymo, and OpenOccupancy-nuScenes. On the Occ3D-nuScenes benchmark, the fusion-based model with ResNet-18 as the image backbone has 21.35M parameters and achieves 51.49 in terms of mean Intersection over Union (mIoU). Furthermore, we propose a multi-stage occupancy-oriented distillation to efficiently transfer knowledge to vision-only OccNet. Extensive experiments on occupancy benchmarks show state-of-the-art precision for both fusion-based and vision-based OccNets. For the demonstration of learning with limited labels, we achieve 94.38\% of the performance (mIoU = 28.38) of a 100\% labeled vision OccNet (mIoU = 30.07) using the same OccNet trained with only 40\% labeled sequences and distillation from the fusion-based OccNet.