UltimateDO: An Efficient Framework to Marry Occupancy Prediction with 3D Object Detection via Channel2height

TL;DR

UltimateDO is a unified framework that efficiently combines occupancy prediction with 3D object detection using 2D convolutions, achieving high precision and fast inference suitable for edge deployment in autonomous driving.

Contribution

It introduces a novel multi-task paradigm reformulated with 2D convolutions, enabling joint occupancy prediction and 3D detection with minimal additional computational cost.

Findings

Achieves fast inference with only 1.1ms overhead.

Facilitates task cooperation for improved accuracy.

Demonstrates effectiveness on nuScenes benchmarks.

Abstract

Occupancy and 3D object detection are characterized as two standard tasks in modern autonomous driving system. In order to deploy them on a series of edge chips with better precision and time-consuming trade-off, contemporary approaches either deploy standalone models for individual tasks, or design a multi-task paradigm with separate heads. However, they might suffer from deployment difficulties (i.e., 3D convolution, transformer and so on) or deficiencies in task coordination. Instead, we argue that a favorable framework should be devised in pursuit of ease deployment on diverse chips and high precision with little time-consuming. Oriented at this, we revisit the paradigm for interaction between 3D object detection and occupancy prediction, reformulate the model with 2D convolution and prioritize the tasks such that each contributes to other. Thus, we propose a method to achieve fast 3D object detection and occupancy prediction (UltimateDO), wherein the light occupancy prediction head in FlashOcc is married to 3D object detection network, with negligible additional timeconsuming of only 1.1ms while facilitating each other. We instantiate UltimateDO on the challenging nuScenes-series benchmarks.