Towards Consistent Object Detection via LiDAR-Camera Synergy

TL;DR

This paper presents an end-to-end framework for consistent object detection using LiDAR and camera data, improving accuracy and robustness in environmental perception for human-machine interaction.

Contribution

It introduces a novel end-to-end COD algorithm that detects objects in both point clouds and images simultaneously and establishes their correlation in a single inference.

Findings

Achieves high detection accuracy on KITTI and DAIR-V2X datasets.

Demonstrates robustness to calibration disturbances.

Provides a new evaluation metric, Consistency Precision (CP).

Abstract

As human-machine interaction continues to evolve, the capacity for environmental perception is becoming increasingly crucial. Integrating the two most common types of sensory data, images, and point clouds, can enhance detection accuracy. Currently, there is no existing model capable of detecting an object's position in both point clouds and images while also determining their corresponding relationship. This information is invaluable for human-machine interactions, offering new possibilities for their enhancement. In light of this, this paper introduces an end-to-end Consistency Object Detection (COD) algorithm framework that requires only a single forward inference to simultaneously obtain an object's position in both point clouds and images and establish their correlation. Furthermore, to assess the accuracy of the object correlation between point clouds and images, this paper proposes a new evaluation metric, Consistency Precision (CP). To verify the effectiveness of the proposed framework, an extensive set of experiments has been conducted on the KITTI and DAIR-V2X datasets. The study also explored how the proposed consistency detection method performs on images when the calibration parameters between images and point clouds are disturbed, compared to existing post-processing methods. The experimental results demonstrate that the proposed method exhibits excellent detection performance and robustness, achieving end-to-end consistency detection. The source code will be made publicly available at https://github.com/xifen523/COD.