VaLID: Verification as Late Integration of Detections for LiDAR-Camera Fusion

TL;DR

VaLID introduces a late-fusion verification method for LiDAR and camera data in vehicle detection, significantly reducing false positives and improving accuracy without dataset-specific tuning.

Contribution

The paper presents a novel model-adaptive late-fusion verification approach, VaLID, that enhances detection accuracy by validating LiDAR detections with camera data using a lightweight neural network.

Findings

Reduces false positives by 63.9% on average

Outperforms individual detectors in 3D average precision

Works effectively with generic, open-vocabulary camera models

Abstract

Vehicle object detection benefits from both LiDAR and camera data, with LiDAR offering superior performance in many scenarios. Fusion of these modalities further enhances accuracy, but existing methods often introduce complexity or dataset-specific dependencies. In our study, we propose a model-adaptive late-fusion method, VaLID, which validates whether each predicted bounding box is acceptable or not. Our method verifies the higher-performing, yet overly optimistic LiDAR model detections using camera detections that are obtained from either specially trained, general, or open-vocabulary models. VaLID uses a lightweight neural verification network trained with a high recall bias to reduce the false predictions made by the LiDAR detector, while still preserving the true ones. Evaluating with multiple combinations of LiDAR and camera detectors on the KITTI dataset, we reduce false positives by an average of 63.9%, thus outperforming the individual detectors on 3D average precision (3DAP). Our approach is model-adaptive and demonstrates state-of-the-art competitive performance even when using generic camera detectors that were not trained specifically for this dataset.