TrafficLoc: Localizing Traffic Surveillance Cameras in 3D Scenes

TL;DR

TrafficLoc is a novel method for accurately localizing traffic cameras in 3D scenes by combining a new dataset, specialized loss functions, and contrastive learning, achieving state-of-the-art results across multiple datasets.

Contribution

The paper introduces TrafficLoc, a new image-to-point cloud registration method with a novel Geometry-guided Attention Loss and contrastive learning, addressing cross-modal matching challenges.

Findings

Achieves up to 86% improvement over SOTA on Carla Intersection

Generalizes well to real-world datasets like KITTI and NuScenes

Sets new SOTA performance on multiple traffic camera datasets

Abstract

We tackle the problem of localizing traffic cameras within a 3D reference map and propose a novel image-to-point cloud registration (I2P) method, TrafficLoc, in a coarse-tofine matching fashion. To overcome the lack of large-scale real-world intersection datasets, we first introduce Carla Intersection, a new simulated dataset with 75 urban and rural intersections in Carla. We find that current I2P methods struggle with cross-modal matching under large viewpoint differences, especially at traffic intersections. TrafficLoc thus employs a novel Geometry-guided Attention Loss (GAL) to focus only on the corresponding geometric regions under different viewpoints during 2D-3D feature fusion. To address feature inconsistency in paired image patch-point groups, we further propose Inter-intra Contrastive Learning (ICL) to enhance separating 2D patch/3D group features within each intra-modality and introduce Dense Training Alignment (DTA) with soft-argmax for improving position regression. Extensive experiments show our TrafficLoc greatly improves the performance over the SOTA I2P methods (up to 86%) on Carla Intersection and generalizes well to real-world data. TrafficLoc also achieves new SOTA performance on KITTI and NuScenes datasets, demonstrating the superiority across both in-vehicle and traffic cameras. Our project page is publicly available at https://tum-luk.github.io/projects/trafficloc/.