End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning

TL;DR

This paper presents an end-to-end deep reinforcement learning approach for active object tracking that generalizes well from simulation to real-world deployment, reducing manual tuning and labeling efforts.

Contribution

The authors introduce a novel end-to-end reinforcement learning framework with environment augmentation and custom rewards for active object tracking, demonstrating successful simulation-to-real transfer.

Findings

The system generalizes to unseen scenarios in simulation.

It can recover from tracking failures effectively.

Successful real-world robot deployment demonstrates practical viability.

Abstract

We study active object tracking, where a tracker takes visual observations (i.e., frame sequences) as input and produces the corresponding camera control signals as output (e.g., move forward, turn left, etc.). Conventional methods tackle tracking and camera control tasks separately, and the resulting system is difficult to tune jointly. These methods also require significant human efforts for image labeling and expensive trial-and-error system tuning in the real world. To address these issues, we propose, in this paper, an end-to-end solution via deep reinforcement learning. A ConvNet-LSTM function approximator is adopted for the direct frame-to-action prediction. We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training. The tracker trained in simulators (ViZDoom and Unreal Engine) demonstrates good generalization behaviors in the case of unseen object moving paths, unseen object appearances, unseen backgrounds, and distracting objects. The system is robust and can restore tracking after occasional lost of the target being tracked. We also find that the tracking ability, obtained solely from simulators, can potentially transfer to real-world scenarios. We demonstrate successful examples of such transfer, via experiments over the VOT dataset and the deployment of a real-world robot using the proposed active tracker trained in simulation.