Active Scout: Multi-Target Tracking Using Neural Radiance Fields in Dense Urban Environments

TL;DR

This paper introduces Active Scout, a novel approach using neural radiance fields (NeRF) for multi-target tracking in dense urban environments, enabling a quadrotor to actively explore and track multiple targets despite occlusions.

Contribution

It presents the first online NeRF-based method for pursuit-evasion in urban settings, combining active perception with dynamic target tracking.

Findings

Successfully tracked 20 stationary targets within 300 steps in simulation.

Maintained a maximum tracking error of 200m for dynamic targets.

Outperformed greedy baseline in occluded, dynamic scenarios.

Abstract

We study pursuit-evasion games in highly occluded urban environments, e.g. tall buildings in a city, where a scout (quadrotor) tracks multiple dynamic targets on the ground. We show that we can build a neural radiance field (NeRF) representation of the city -- online -- using RGB and depth images from different vantage points. This representation is used to calculate the information gain to both explore unknown parts of the city and track the targets -- thereby giving a completely first-principles approach to actively tracking dynamic targets. We demonstrate, using a custom-built simulator using Open Street Maps data of Philadelphia and New York City, that we can explore and locate 20 stationary targets within 300 steps. This is slower than a greedy baseline, which does not use active perception. But for dynamic targets that actively hide behind occlusions, we show that our approach maintains, at worst, a tracking error of 200m; the greedy baseline can have a tracking error as large as 600m. We observe a number of interesting properties in the scout's policies, e.g., it switches its attention to track a different target periodically, as the quality of the NeRF representation improves over time, the scout also becomes better in terms of target tracking. Code is available at https://github.com/grasp-lyrl/ActiveScout.