Learning-Based Sensor Scheduling for Delay-Aware and Stable Remote State Estimation

TL;DR

This paper introduces a delay-aware sensor scheduling framework for remote state estimation that optimizes information gain and energy efficiency, using reinforcement learning to adaptively manage delays and sensor heterogeneity.

Contribution

It presents a unified delay-aware model, an efficient delayed measurement update, and a PPO-based scheduler that learns optimal policies without prior delay models.

Findings

Achieves lower estimation error compared to baselines.

Maintains robustness to measurement and noise variations.

Requires no prior delay model for scheduling decisions.

Abstract

Unpredictable sensor-to-estimator delays fundamentally distort what matters for wireless remote state estimation: not just freshness, but how delay interacts with sensor informativeness and energy efficiency. In this paper, we present a unified, delay-aware framework that models this coupling explicitly and quantifies a delay-dependent information gain, motivating an information-per-joule scheduling objective beyond age of information proxies (AoI). To this end, we first introduce an efficient posterior-fusion update that incorporates delayed measurements without state augmentation, providing a consistent approximation to optimal delayed Kalman updates, and then derive tractable stability conditions ensuring that bounded estimation error is achievable under stochastic, delayed scheduling. This conditions highlight the need for unstable modes to be observable across sensors. Building on this foundation, we cast scheduling as a Markov decision process and develop a proximal policy optimization (PPO) scheduler that learns directly from interaction, requires no prior delay model, and explicitly trades off estimation accuracy, freshness, sensor heterogeneity, and transmission energy through normalized rewards. In simulations with heterogeneous sensors, realistic link-energy models, and random delays, the proposed method learns stably and consistently achieves lower estimation error at comparable energy than random scheduling and strong RL baselines (DQN, A2C), while remaining robust to variations in measurement availability and process/measurement noise.