Towards Distraction-Robust Active Visual Tracking

TL;DR

This paper introduces a novel multi-agent game framework to improve the robustness of active visual trackers against distractors by learning diverse distracting behaviors and employing advanced training strategies.

Contribution

It proposes a mixed cooperative-competitive multi-agent game and new learning methods to enhance distraction-robustness in visual tracking.

Findings

Tracker achieves improved distraction-robustness in experiments.

Multi-agent game exposes tracker weaknesses effectively.

Proposed methods generalize well to unseen environments.

Abstract

In active visual tracking, it is notoriously difficult when distracting objects appear, as distractors often mislead the tracker by occluding the target or bringing a confusing appearance. To address this issue, we propose a mixed cooperative-competitive multi-agent game, where a target and multiple distractors form a collaborative team to play against a tracker and make it fail to follow. Through learning in our game, diverse distracting behaviors of the distractors naturally emerge, thereby exposing the tracker's weakness, which helps enhance the distraction-robustness of the tracker. For effective learning, we then present a bunch of practical methods, including a reward function for distractors, a cross-modal teacher-student learning strategy, and a recurrent attention mechanism for the tracker. The experimental results show that our tracker performs desired distraction-robust active visual tracking and can be well generalized to unseen environments. We also show that the multi-agent game can be used to adversarially test the robustness of trackers.