From Detection to Association: Learning Discriminative Object Embeddings for Multi-Object Tracking

TL;DR

This paper introduces FDTA, a framework that improves multi-object tracking by refining object embeddings to better distinguish instances across frames, addressing limitations of existing end-to-end methods.

Contribution

It proposes a novel feature refinement framework with spatial, temporal, and identity adapters to enhance discriminative object embeddings for tracking.

Findings

Achieves state-of-the-art results on multiple MOT benchmarks.

Effectively improves association accuracy over existing methods.

Demonstrates the importance of discriminative embeddings for multi-object tracking.

Abstract

End-to-end multi-object tracking (MOT) methods have recently achieved remarkable progress by unifying detection and association within a single framework. Despite their strong detection performance, these methods suffer from relatively low association accuracy. Through detailed analysis, we observe that object embeddings produced by the shared DETR architecture display excessively high inter-object similarity, as it emphasizes only category-level discrimination within single frames. In contrast, tracking requires instance-level distinction across frames with spatial and temporal continuity, for which current end-to-end approaches insufficiently optimize object embeddings. To address this, we introduce FDTA (From Detection to Association), an explicit feature refinement framework that enhances object discriminativeness across three complementary perspectives. Specifically, we introduce a Spatial Adapter (SA) to integrate depth-aware cues for spatial continuity, a Temporal Adapter (TA) to aggregate historical information for temporal dependencies, and an Identity Adapter (IA) to leverage quality-aware contrastive learning for instance-level separability. Extensive experiments demonstrate that FDTA achieves state-of-the-art performance on multiple challenging MOT benchmarks, including DanceTrack, SportsMOT, and BFT, highlighting the effectiveness of our proposed discriminative embedding enhancement strategy. The code is available at https://github.com/Spongebobbbbbbbb/FDTA.