Progressive Coordinate Transforms for Monocular 3D Object Detection

TL;DR

This paper introduces Progressive Coordinate Transforms (PCT), a lightweight method that improves monocular 3D object detection by refining object localization through a confidence-aware loss and semantic features, achieving superior results on KITTI and Waymo datasets.

Contribution

The paper proposes a novel, lightweight coordinate refinement approach called PCT that enhances monocular 3D detection accuracy by progressively improving localization predictions.

Findings

Achieves state-of-the-art results on KITTI and Waymo benchmarks.

Demonstrates generalization across various coordinate-based 3D detection frameworks.

Provides a simple yet effective localization refinement mechanism.

Abstract

Recognizing and localizing objects in the 3D space is a crucial ability for an AI agent to perceive its surrounding environment. While significant progress has been achieved with expensive LiDAR point clouds, it poses a great challenge for 3D object detection given only a monocular image. While there exist different alternatives for tackling this problem, it is found that they are either equipped with heavy networks to fuse RGB and depth information or empirically ineffective to process millions of pseudo-LiDAR points. With in-depth examination, we realize that these limitations are rooted in inaccurate object localization. In this paper, we propose a novel and lightweight approach, dubbed {\em Progressive Coordinate Transforms} (PCT) to facilitate learning coordinate representations. Specifically, a localization boosting mechanism with confidence-aware loss is introduced to progressively refine the localization prediction. In addition, semantic image representation is also exploited to compensate for the usage of patch proposals. Despite being lightweight and simple, our strategy leads to superior improvements on the KITTI and Waymo Open Dataset monocular 3D detection benchmarks. At the same time, our proposed PCT shows great generalization to most coordinate-based 3D detection frameworks. The code is available at: https://github.com/amazon-research/progressive-coordinate-transforms .