Adapting SAM 2 for Visual Object Tracking: 1st Place Solution for MMVPR Challenge Multi-Modal Tracking

TL;DR

This paper adapts the Segment Anything Model 2 (SAM2) for visual object tracking, achieving top performance in a multi-modal tracking challenge by integrating optimizations and demonstrating its effectiveness.

Contribution

The paper introduces a novel adaptation of SAM2 for VOT, incorporating specific enhancements to improve tracking performance in multi-modal datasets.

Findings

Achieved first place with an AUC score of 89.4 in the ICPR 2024 challenge.

Demonstrated the effectiveness of SAM2 adaptation for multi-modal visual tracking.

Provided comprehensive analysis of the proposed method's performance.

Abstract

We present an effective approach for adapting the Segment Anything Model 2 (SAM2) to the Visual Object Tracking (VOT) task. Our method leverages the powerful pre-trained capabilities of SAM2 and incorporates several key techniques to enhance its performance in VOT applications. By combining SAM2 with our proposed optimizations, we achieved a first place AUC score of 89.4 on the 2024 ICPR Multi-modal Object Tracking challenge, demonstrating the effectiveness of our approach. This paper details our methodology, the specific enhancements made to SAM2, and a comprehensive analysis of our results in the context of VOT solutions along with the multi-modality aspect of the dataset.