Towards Unified 3D Object Detection via Algorithm and Data Unification

TL;DR

This paper introduces a unified approach for 3D object detection across indoor and outdoor scenes, leveraging algorithmic innovations and multi-modal data, and presents a new benchmark to evaluate such models.

Contribution

It proposes a novel monocular 3D detector with a two-stage BEV architecture, a unified multi-modal detector, and a new benchmark for comprehensive evaluation.

Findings

Multi-modal data improves detection robustness.

The proposed strategies enhance model performance across diverse scenarios.

The unified benchmark facilitates fair comparison of 3D detection methods.

Abstract

Realizing unified 3D object detection, including both indoor and outdoor scenes, holds great importance in applications like robot navigation. However, involving various scenarios of data to train models poses challenges due to their significantly distinct characteristics, \eg, diverse geometry properties and heterogeneous domain distributions. In this work, we propose to address the challenges from two perspectives, the algorithm perspective and data perspective. In terms of the algorithm perspective, we first build a monocular 3D object detector based on the bird's-eye-view (BEV) detection paradigm, where the explicit feature projection is beneficial to addressing the geometry learning ambiguity. In this detector, we split the classical BEV detection architecture into two stages and propose an uneven BEV grid design to handle the convergence instability caused by geometry difference between scenarios. Besides, we develop a sparse BEV feature projection strategy to reduce the computational cost and a unified domain alignment method to handle heterogeneous domains. From the data perspective, we propose to incorporate depth information to improve training robustness. Specifically, we build the first unified multi-modal 3D object detection benchmark MM-Omni3D and extend the aforementioned monocular detector to its multi-modal version, which is the first unified multi-modal 3D object detector. We name the designed monocular and multi-modal detectors as UniMODE and MM-UniMODE, respectively. The experimental results reveal several insightful findings highlighting the benefits of multi-modal data and confirm the effectiveness of all the proposed strategies.