Beyond standard benchmarks: Parameterizing performance evaluation in visual object tracking

TL;DR

This paper introduces a novel evaluation framework for visual object tracking that uses omnidirectional videos to generate realistic, parameterized motion scenarios, addressing limitations of standard benchmarks.

Contribution

It presents a new dataset, annotation method, and evaluation system that better captures the influence of object-to-camera motion on tracking performance.

Findings

Standard benchmarks lack detailed motion annotations.

The proposed approach provides more realistic and controllable evaluation scenarios.

Analysis confirms the effectiveness of parameterized motion patterns in evaluating trackers.

Abstract

Object-to-camera motion produces a variety of apparent motion patterns that significantly affect performance of short-term visual trackers. Despite being crucial for designing robust trackers, their influence is poorly explored in standard benchmarks due to weakly defined, biased and overlapping attribute annotations. In this paper we propose to go beyond pre-recorded benchmarks with post-hoc annotations by presenting an approach that utilizes omnidirectional videos to generate realistic, consistently annotated, short-term tracking scenarios with exactly parameterized motion patterns. We have created an evaluation system, constructed a fully annotated dataset of omnidirectional videos and the generators for typical motion patterns. We provide an in-depth analysis of major tracking paradigms which is complementary to the standard benchmarks and confirms the expressiveness of our evaluation approach.