Uni3DETR: Unified 3D Detection Transformer

TL;DR

Uni3DETR is a unified 3D detection transformer that effectively handles indoor and outdoor point cloud data within a single framework, demonstrating strong generalization and consistent performance across diverse scenes.

Contribution

The paper introduces Uni3DETR, a novel unified 3D detection framework that employs point-voxel interaction, mixture of query points, and decoupled IoU for improved cross-scene 3D detection.

Findings

Excellent performance on both indoor and outdoor datasets.

Strong generalization ability across heterogeneous scenes.

Outperforms specialized detectors in diverse environments.

Abstract

Existing point cloud based 3D detectors are designed for the particular scene, either indoor or outdoor ones. Because of the substantial differences in object distribution and point density within point clouds collected from various environments, coupled with the intricate nature of 3D metrics, there is still a lack of a unified network architecture that can accommodate diverse scenes. In this paper, we propose Uni3DETR, a unified 3D detector that addresses indoor and outdoor 3D detection within the same framework. Specifically, we employ the detection transformer with point-voxel interaction for object prediction, which leverages voxel features and points for cross-attention and behaves resistant to the discrepancies from data. We then propose the mixture of query points, which sufficiently exploits global information for dense small-range indoor scenes and local information for large-range sparse outdoor ones. Furthermore, our proposed decoupled IoU provides an easy-to-optimize training target for localization by disentangling the xy and z space. Extensive experiments validate that Uni3DETR exhibits excellent performance consistently on both indoor and outdoor 3D detection. In contrast to previous specialized detectors, which may perform well on some particular datasets but suffer a substantial degradation on different scenes, Uni3DETR demonstrates the strong generalization ability under heterogeneous conditions (Fig. 1). Codes are available at \href{https://github.com/zhenyuw16/Uni3DETR}{https://github.com/zhenyuw16/Uni3DETR}.