Quantum Inverse Contextual Vision Transformers (Q-ICVT): A New Frontier in 3D Object Detection for AVs

TL;DR

Q-ICVT introduces a quantum-inspired two-stage fusion method for 3D object detection in autonomous vehicles, effectively integrating LiDAR and camera data to improve detection accuracy, especially at long distances.

Contribution

The paper proposes a novel quantum-inspired reversible vision transformer (GAT) and a local feature fusion module (SELF) for enhanced multi-modal data fusion in AVs.

Findings

Achieves 82.54% mAPH on Waymo dataset, surpassing previous methods.

Demonstrates the effectiveness of quantum-inspired transformers in sensor data fusion.

Provides ablation studies validating the impact of each component.

Abstract

The field of autonomous vehicles (AVs) predominantly leverages multi-modal integration of LiDAR and camera data to achieve better performance compared to using a single modality. However, the fusion process encounters challenges in detecting distant objects due to the disparity between the high resolution of cameras and the sparse data from LiDAR. Insufficient integration of global perspectives with local-level details results in sub-optimal fusion performance.To address this issue, we have developed an innovative two-stage fusion process called Quantum Inverse Contextual Vision Transformers (Q-ICVT). This approach leverages adiabatic computing in quantum concepts to create a novel reversible vision transformer known as the Global Adiabatic Transformer (GAT). GAT aggregates sparse LiDAR features with semantic features in dense images for cross-modal integration in a global form. Additionally, the Sparse Expert of Local Fusion (SELF) module maps the sparse LiDAR 3D proposals and encodes position information of the raw point cloud onto the dense camera feature space using a gating point fusion approach. Our experiments show that Q-ICVT achieves an mAPH of 82.54 for L2 difficulties on the Waymo dataset, improving by 1.88% over current state-of-the-art fusion methods. We also analyze GAT and SELF in ablation studies to highlight the impact of Q-ICVT. Our code is available at https://github.com/sanjay-810/Qicvt Q-ICVT