CalibFree: Self-Supervised View Feature Separation for Calibration-Free Multi-Camera Multi-Object Tracking

TL;DR

CalibFree introduces a self-supervised, calibration-free framework for multi-camera multi-object tracking that improves accuracy and adaptability without manual labeling or calibration.

Contribution

It proposes a novel self-supervised learning method that separates view-agnostic and view-specific features for robust multi-camera tracking without calibration.

Findings

3% improvement in overall accuracy on MMP-MvMHAT dataset

7.5% increase in average F1 score over state-of-the-art methods

Demonstrates superior over-time tracking and cross-view performance

Abstract

Multi-camera multi-object tracking (MCMOT) faces significant challenges in maintaining consistent object identities across varying camera perspectives, particularly when precise calibration and extensive annotations are required. In this paper, we present CalibFree, a self-supervised representation learning framework that does not need any calibration or manual labeling for the MCMOT task. By promoting feature separation between view-agnostic and view-specific representations through single-view distillation and cross-view reconstruction, our method adapts to complex, dynamic scenarios with minimal overhead. Experiments on the MMP-MvMHAT dataset show a 3% improvement in overall accuracy and a 7.5% increase in the average F1 score over state-of-the-art approaches, confirming the effectiveness of our calibration-free design. Moreover, on the more diverse MvMHAT dataset, our approach demonstrates superior over-time tracking and strong cross-view performance, highlighting its adaptability to a wide range of camera configurations. Code will be publicly available upon acceptance.