Target-Following Double Deep Q-Networks for UAVs

TL;DR

This paper introduces TF-DDQN, a deep reinforcement learning approach for UAV target tracking in complex environments, along with a new evaluation scheme to benchmark tracking performance under diverse conditions.

Contribution

It proposes a novel DDQN-based method for UAV target tracking and a comprehensive evaluation scheme for benchmarking in real-world scenarios.

Findings

TF-DDQN outperforms standard baselines in complex environments

The evaluation scheme effectively measures tracking efficacy across diverse parameters

The approach handles environmental changes and target motion uncertainty

Abstract

Target tracking in unknown real-world environments in the presence of obstacles and target motion uncertainty demand agents to develop an intrinsic understanding of the environment in order to predict the suitable actions to be taken at each time step. This task requires the agents to maximize the visibility of the mobile target maneuvering randomly in a network of roads by learning a policy that takes into consideration the various aspects of a real-world environment. In this paper, we propose a DDQN-based extension to the state-of-the-art in target tracking using a UAV TF-DQN, that we call TF-DDQN, that isolates the value estimation and evaluation steps. Additionally, in order to carefully benchmark the performance of any given target tracking algorithm, we introduce a novel target tracking evaluation scheme that quantifies its efficacy in terms of a wide set of diverse parameters. To replicate the real-world setting, we test our approach against standard baselines for the task of target tracking in complex environments with varying drift conditions and changes in environmental configuration.