SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking

TL;DR

SiamCAR introduces a fully convolutional Siamese network for visual tracking that is proposal and anchor free, simplifying the process and achieving state-of-the-art performance efficiently.

Contribution

It presents a novel end-to-end per-pixel tracking framework that eliminates the need for region proposals and anchors, reducing hyper-parameter tuning and human intervention.

Findings

Achieves leading performance on multiple benchmarks.

Operates in real-time with high accuracy.

Simplifies tracking by removing region proposals and anchors.

Abstract

By decomposing the visual tracking task into two subproblems as classification for pixel category and regression for object bounding box at this pixel, we propose a novel fully convolutional Siamese network to solve visual tracking end-to-end in a per-pixel manner. The proposed framework SiamCAR consists of two simple subnetworks: one Siamese subnetwork for feature extraction and one classification-regression subnetwork for bounding box prediction. Our framework takes ResNet-50 as backbone. Different from state-of-the-art trackers like Siamese-RPN, SiamRPN++ and SPM, which are based on region proposal, the proposed framework is both proposal and anchor free. Consequently, we are able to avoid the tricky hyper-parameter tuning of anchors and reduce human intervention. The proposed framework is simple, neat and effective. Extensive experiments and comparisons with state-of-the-art trackers are conducted on many challenging benchmarks like GOT-10K, LaSOT, UAV123 and OTB-50. Without bells and whistles, our SiamCAR achieves the leading performance with a considerable real-time speed.