Long and Short Memory Balancing in Visual Co-Tracking using Q-Learning

TL;DR

This paper introduces an active co-tracking framework that uses reinforcement learning to dynamically decide when to consult auxiliary classifiers, improving tracking accuracy and efficiency in visual co-tracking tasks.

Contribution

It presents a novel reinforcement learning-based policy for effective information exchange between classifiers in visual co-tracking, addressing computational complexity and tracking challenges.

Findings

Outperforms existing tracking-by-detection methods

Reduces computational complexity in co-tracking

Improves robustness against occlusions and label errors

Abstract

Employing one or more additional classifiers to break the self-learning loop in tracing-by-detection has gained considerable attention. Most of such trackers merely utilize the redundancy to address the accumulating label error in the tracking loop, and suffer from high computational complexity as well as tracking challenges that may interrupt all classifiers (e.g. temporal occlusions). We propose the active co-tracking framework, in which the main classifier of the tracker labels samples of the video sequence, and only consults auxiliary classifier when it is uncertain. Based on the source of the uncertainty and the differences of two classifiers (e.g. accuracy, speed, update frequency, etc.), different policies should be taken to exchange the information between two classifiers. Here, we introduce a reinforcement learning approach to find the appropriate policy by considering the state of the tracker in a specific sequence. The proposed method yields promising results in comparison to the best tracking-by-detection approaches.