Are we Missing Confidence in Pseudo-LiDAR Methods for Monocular 3D Object Detection?

TL;DR

This paper critically examines the bias in reported results of Pseudo-LiDAR methods for monocular 3D detection, reveals the impact of dataset overlap, and proposes a new architecture with 3D confidence estimation to achieve state-of-the-art performance.

Contribution

It identifies and rectifies biases in existing evaluations and introduces a novel deep architecture with 3D confidence prediction for improved monocular 3D detection.

Findings

Validation results are biased due to dataset overlaps.

Removing overlaps does not fully eliminate bias, making test set results more reliable.

The proposed architecture achieves state-of-the-art results on KITTI3D.

Abstract

Pseudo-LiDAR-based methods for monocular 3D object detection have received considerable attention in the community due to the performance gains exhibited on the KITTI3D benchmark, in particular on the commonly reported validation split. This generated a distorted impression about the superiority of Pseudo-LiDAR-based (PL-based) approaches over methods working with RGB images only. Our first contribution consists in rectifying this view by pointing out and showing experimentally that the validation results published by PL-based methods are substantially biased. The source of the bias resides in an overlap between the KITTI3D object detection validation set and the training/validation sets used to train depth predictors feeding PL-based methods. Surprisingly, the bias remains also after geographically removing the overlap. This leaves the test set as the only reliable set for comparison, where published PL-based methods do not excel. Our second contribution brings PL-based methods back up in the ranking with the design of a novel deep architecture which introduces a 3D confidence prediction module. We show that 3D confidence estimation techniques derived from RGB-only 3D detection approaches can be successfully integrated into our framework and, more importantly, that improved performance can be obtained with a newly designed 3D confidence measure, leading to state-of-the-art performance on the KITTI3D benchmark.