MITracker: Multi-View Integration for Visual Object Tracking

TL;DR

MITracker is a novel multi-view object tracking method that integrates features from multiple viewpoints using 3D volume transformation and attention mechanisms, significantly improving tracking stability and accuracy.

Contribution

We introduce MITracker, a new multi-view tracking algorithm that transforms 2D features into 3D space and employs geometric attention for enhanced multi-view integration.

Findings

MITracker achieves state-of-the-art results on MVTrack and GMTD datasets.

The new MVTrack dataset contains 234K annotated frames across various scenes.

MITracker effectively handles arbitrary viewpoints and long video sequences.

Abstract

Multi-view object tracking (MVOT) offers promising solutions to challenges such as occlusion and target loss, which are common in traditional single-view tracking. However, progress has been limited by the lack of comprehensive multi-view datasets and effective cross-view integration methods. To overcome these limitations, we compiled a Multi-View object Tracking (MVTrack) dataset of 234K high-quality annotated frames featuring 27 distinct objects across various scenes. In conjunction with this dataset, we introduce a novel MVOT method, Multi-View Integration Tracker (MITracker), to efficiently integrate multi-view object features and provide stable tracking outcomes. MITracker can track any object in video frames of arbitrary length from arbitrary viewpoints. The key advancements of our method over traditional single-view approaches come from two aspects: (1) MITracker transforms 2D image features into a 3D feature volume and compresses it into a bird's eye view (BEV) plane, facilitating inter-view information fusion; (2) we propose an attention mechanism that leverages geometric information from fused 3D feature volume to refine the tracking results at each view. MITracker outperforms existing methods on the MVTrack and GMTD datasets, achieving state-of-the-art performance. The code and the new dataset will be available at https://mii-laboratory.github.io/MITracker/.