Recursive Least-Squares Estimator-Aided Online Learning for Visual Tracking

TL;DR

This paper introduces a recursive least-squares estimator-based online learning method for visual tracking that enhances model adaptation and memory retention without offline training, improving performance on challenging benchmarks.

Contribution

It proposes a simple online learning approach using recursive least-squares estimation that eliminates the need for offline training and helps prevent catastrophic forgetting.

Findings

Improves tracking accuracy on multiple benchmarks.

Reduces computational cost compared to meta-learning methods.

Enhances model memory retention during online adaptation.

Abstract

Tracking visual objects from a single initial exemplar in the testing phase has been broadly cast as a one-/few-shot problem, i.e., one-shot learning for initial adaptation and few-shot learning for online adaptation. The recent few-shot online adaptation methods incorporate the prior knowledge from large amounts of annotated training data via complex meta-learning optimization in the offline phase. This helps the online deep trackers to achieve fast adaptation and reduce overfitting risk in tracking. In this paper, we propose a simple yet effective recursive least-squares estimator-aided online learning approach for few-shot online adaptation without requiring offline training. It allows an in-built memory retention mechanism for the model to remember the knowledge about the object seen before, and thus the seen data can be safely removed from training. This also bears certain similarities to the emerging continual learning field in preventing catastrophic forgetting. This mechanism enables us to unveil the power of modern online deep trackers without incurring too much extra computational cost. We evaluate our approach based on two networks in the online learning families for tracking, i.e., multi-layer perceptrons in RT-MDNet and convolutional neural networks in DiMP. The consistent improvements on several challenging tracking benchmarks demonstrate its effectiveness and efficiency.