Exploit the Connectivity: Multi-Object Tracking with TrackletNet

TL;DR

This paper introduces TrackletNet Tracker (TNT), a novel multi-object tracking method that combines appearance and temporal features using a graph model and a multi-scale neural network, improving tracking robustness.

Contribution

The paper presents a unified framework that integrates appearance and temporal information for MOT, using a graph model and a specialized neural network for similarity measurement.

Findings

Achieves promising results on MOT16 and MOT17 benchmarks.

Effectively handles occlusion and camera movement challenges.

Outperforms several state-of-the-art methods in accuracy.

Abstract

Multi-object tracking (MOT) is an important and practical task related to both surveillance systems and moving camera applications, such as autonomous driving and robotic vision. However, due to unreliable detection, occlusion and fast camera motion, tracked targets can be easily lost, which makes MOT very challenging. Most recent works treat tracking as a re-identification (Re-ID) task, but how to combine appearance and temporal features is still not well addressed. In this paper, we propose an innovative and effective tracking method called TrackletNet Tracker (TNT) that combines temporal and appearance information together as a unified framework. First, we define a graph model which treats each tracklet as a vertex. The tracklets are generated by appearance similarity with CNN features and intersection-over-union (IOU) with epipolar constraints to compensate camera movement between adjacent frames. Then, for every pair of two tracklets, the similarity is measured by our designed multi-scale TrackletNet. Afterwards, the tracklets are clustered into groups which represent individual object IDs. Our proposed TNT has the ability to handle most of the challenges in MOT, and achieve promising results on MOT16 and MOT17 benchmark datasets compared with other state-of-the-art methods.