Benchmarking SAM2-based Trackers on FMOX

TL;DR

This paper benchmarks SAM2-based object trackers on datasets with fast-moving objects to evaluate their performance and limitations in challenging scenarios.

Contribution

It introduces a benchmark for SAM2-based trackers on FMO datasets and provides detailed insights into their performance and limitations.

Findings

DAM4SAM and SAMURAI perform well on challenging sequences

Trackers show limitations with fast-moving objects

Benchmarking reveals specific strengths and weaknesses

Abstract

Several object tracking pipelines extending Segment Anything Model 2 (SAM2) have been proposed in the past year, where the approach is to follow and segment the object from a single exemplar template provided by the user on a initialization frame. We propose to benchmark these high performing trackers (SAM2, EfficientTAM, DAM4SAM and SAMURAI) on datasets containing fast moving objects (FMO) specifically designed to be challenging for tracking approaches. The goal is to understand better current limitations in state-of-the-art trackers by providing more detailed insights on the behavior of these trackers. We show that overall the trackers DAM4SAM and SAMURAI perform well on more challenging sequences.