Parallel Tracking and Verifying: A Framework for Real-Time and High Accuracy Visual Tracking

TL;DR

This paper introduces PTAV, a parallel tracking and verifying framework that combines real-time speed with high accuracy by using separate threads for tracking and verification, improving performance on standard benchmarks.

Contribution

The novel PTAV framework leverages parallel processing with a tracker and verifier working asynchronously, enhancing real-time tracking accuracy beyond existing methods.

Findings

Achieves state-of-the-art accuracy among real-time trackers

Outperforms many deep learning-based solutions in benchmarks

Flexible framework with potential for further improvements

Abstract

Being intensively studied, visual tracking has seen great recent advances in either speed (e.g., with correlation filters) or accuracy (e.g., with deep features). Real-time and high accuracy tracking algorithms, however, remain scarce. In this paper we study the problem from a new perspective and present a novel parallel tracking and verifying (PTAV) framework, by taking advantage of the ubiquity of multi-thread techniques and borrowing from the success of parallel tracking and mapping in visual SLAM. Our PTAV framework typically consists of two components, a tracker T and a verifier V, working in parallel on two separate threads. The tracker T aims to provide a super real-time tracking inference and is expected to perform well most of the time; by contrast, the verifier V checks the tracking results and corrects T when needed. The key innovation is that, V does not work on every frame but only upon the requests from T; on the other end, T may adjust the tracking according to the feedback from V. With such collaboration, PTAV enjoys both the high efficiency provided by T and the strong discriminative power by V. In our extensive experiments on popular benchmarks including OTB2013, OTB2015, TC128 and UAV20L, PTAV achieves the best tracking accuracy among all real-time trackers, and in fact performs even better than many deep learning based solutions. Moreover, as a general framework, PTAV is very flexible and has great rooms for improvement and generalization.