Sensor Control for Information Gain in Dynamic, Sparse and Partially Observed Environments

TL;DR

This paper introduces an advanced reinforcement learning approach for autonomous sensor control that maximizes information gain in dynamic, sparse, and partially observed environments, specifically applied to RF spectrum monitoring.

Contribution

It extends the Deep Anticipatory Network framework with novel exploration strategies and hybrid neural layers, and introduces a model-based RL variant for limited sampling scenarios.

Findings

Outperforms standard DAN in simulated RF environments.

More flexible and robust than baseline agents.

Adaptable to non-stationary emission environments.

Abstract

We present an approach for autonomous sensor control for information gathering under partially observable, dynamic and sparsely sampled environments that maximizes information about entities present in that space. We describe our approach for the task of Radio-Frequency (RF) spectrum monitoring, where the goal is to search for and track unknown, dynamic signals in the environment. To this end, we extend the Deep Anticipatory Network (DAN) Reinforcement Learning (RL) framework by (1) improving exploration in sparse, non-stationary environments using a novel information gain reward, and (2) scaling up the control space and enabling the monitoring of complex, dynamic activity patterns using hybrid convolutional-recurrent neural layers. We also extend this problem to situations in which sampling from the intended RF spectrum/field is limited and propose a model-based version of the original RL algorithm that fine-tunes the controller via a model that is iteratively improved from the limited field sampling. Results in simulated RF environments of differing complexity show that our system outperforms the standard DAN architecture and is more flexible and robust than baseline expert-designed agents. We also show that it is adaptable to non-stationary emission environments.