Occupancy as Set of Points

TL;DR

This paper introduces a flexible point-based 3D occupancy prediction method called OSP, which outperforms traditional volume-based approaches and can be integrated with them for improved scene understanding from multi-view images.

Contribution

The paper proposes a novel point-based framework, OSP, for 3D occupancy prediction that enhances performance and flexibility over existing volume-based methods.

Findings

OSP achieves superior accuracy on Occ3D nuScenes benchmark.

The method demonstrates high adaptability in training and inference.

OSP can be integrated with volume-based methods to improve their effectiveness.

Abstract

In this paper, we explore a novel point representation for 3D occupancy prediction from multi-view images, which is named Occupancy as Set of Points. Existing camera-based methods tend to exploit dense volume-based representation to predict the occupancy of the whole scene, making it hard to focus on the special areas or areas out of the perception range. In comparison, we present the Points of Interest (PoIs) to represent the scene and propose OSP, a novel framework for point-based 3D occupancy prediction. Owing to the inherent flexibility of the point-based representation, OSP achieves strong performance compared with existing methods and excels in terms of training and inference adaptability. It extends beyond traditional perception boundaries and can be seamlessly integrated with volume-based methods to significantly enhance their effectiveness. Experiments on the Occ3D nuScenes occupancy benchmark show that OSP has strong performance and flexibility. Code and models are available at \url{https://github.com/hustvl/osp}.