Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting

TL;DR

This paper introduces a novel spatiotemporal decoupling approach for vision-based 3D occupancy forecasting that improves accuracy and efficiency by addressing spatial and temporal biases, utilizing a new representation and a specialized network.

Contribution

The paper proposes a decoupled 3D occupancy representation and an efficient multi-head network, achieving state-of-the-art accuracy and speed in occupancy forecasting.

Findings

Surpasses existing methods in accuracy and efficiency.

Achieves inference time of 82.33ms on a single GPU.

Introduces the conditional IoU metric for better performance assessment.

Abstract

The task of occupancy forecasting (OCF) involves utilizing past and present perception data to predict future occupancy states of autonomous vehicle surrounding environments, which is critical for downstream tasks such as obstacle avoidance and path planning. Existing 3D OCF approaches struggle to predict plausible spatial details for movable objects and suffer from slow inference speeds due to neglecting the bias and uneven distribution of changing occupancy states in both space and time. In this paper, we propose a novel spatiotemporal decoupling vision-based paradigm to explicitly tackle the bias and achieve both effective and efficient 3D OCF. To tackle spatial bias in empty areas, we introduce a novel spatial representation that decouples the conventional dense 3D format into 2D bird's-eye view (BEV) occupancy with corresponding height values, enabling 3D OCF derived only from 2D predictions thus enhancing efficiency. To reduce temporal bias on static voxels, we design temporal decoupling to improve end-to-end OCF by temporally associating instances via predicted flows. We develop an efficient multi-head network EfficientOCF to achieve 3D OCF with our devised spatiotemporally decoupled representation. A new metric, conditional IoU (C-IoU), is also introduced to provide a robust 3D OCF performance assessment, especially in datasets with missing or incomplete annotations. The experimental results demonstrate that EfficientOCF surpasses existing baseline methods on accuracy and efficiency, achieving state-of-the-art performance with a fast inference time of 82.33ms with a single GPU. Our code will be released as open source.