Far3D: Expanding the Horizon for Surround-view 3D Object Detection

TL;DR

Far3D introduces a novel sparse query-based framework for long-range surround-view 3D object detection, effectively addressing computational and convergence challenges, and achieves state-of-the-art results on Argoverse 2 and nuScenes datasets.

Contribution

The paper proposes a new sparse query-based approach with adaptive queries, perspective-aware aggregation, and range-modulated denoising for improved long-range 3D detection.

Findings

Achieves state-of-the-art performance on Argoverse 2 at 150 meters

Outperforms previous methods on nuScenes dataset

Effectively handles long-range detection challenges

Abstract

Recently 3D object detection from surround-view images has made notable advancements with its low deployment cost. However, most works have primarily focused on close perception range while leaving long-range detection less explored. Expanding existing methods directly to cover long distances poses challenges such as heavy computation costs and unstable convergence. To address these limitations, this paper proposes a novel sparse query-based framework, dubbed Far3D. By utilizing high-quality 2D object priors, we generate 3D adaptive queries that complement the 3D global queries. To efficiently capture discriminative features across different views and scales for long-range objects, we introduce a perspective-aware aggregation module. Additionally, we propose a range-modulated 3D denoising approach to address query error propagation and mitigate convergence issues in long-range tasks. Significantly, Far3D demonstrates SoTA performance on the challenging Argoverse 2 dataset, covering a wide range of 150 meters, surpassing several LiDAR-based approaches. Meanwhile, Far3D exhibits superior performance compared to previous methods on the nuScenes dataset. The code is available at https://github.com/megvii-research/Far3D.