Multi-Camera Multi-Person Association using Transformer-Based Dense Pixel Correspondence Estimation and Detection-Based Masking

TL;DR

This paper introduces a novel multi-camera multi-target association method using Transformer-based dense pixel correspondence and detection masking, achieving high accuracy in similar viewpoints but needing improvement for diverse angles.

Contribution

The paper presents a new Transformer-based dense pixel correspondence approach combined with detection masking for multi-camera multi-target association.

Findings

Performs well with similar camera viewpoints

Shows room for improvement with diverse camera angles

Utilizes Hungarian algorithm for optimal association

Abstract

Multi-camera Association (MCA) is the task of identifying objects and individuals across camera views and is an active research topic, given its numerous applications across robotics, surveillance, and agriculture. We investigate a novel multi-camera multi-target association algorithm based on dense pixel correspondence estimation with a Transformer-based architecture and underlying detection-based masking. After the algorithm generates a set of corresponding keypoints and their respective confidence levels between every pair of detections in the camera views are computed, an affinity matrix is determined containing the probabilities of matches between each pair. Finally, the Hungarian algorithm is applied to generate an optimal assignment matrix with all the predicted associations between the camera views. Our method is evaluated on the WILDTRACK Seven-Camera HD Dataset, a high-resolution dataset containing footage of walking pedestrians as well as precise annotations and camera calibrations. Our results conclude that the algorithm performs exceptionally well associating pedestrians on camera pairs that are positioned close to each other and observe the scene from similar perspectives. On camera pairs with orientations that are drastically different in distance or angle, there is still significant room for improvement.