Stability Enforced Bandit Algorithms for Channel Selection in Remote State Estimation of Gauss-Markov Processes

TL;DR

This paper develops bandit-based algorithms for remote state estimation of Gauss-Markov processes, ensuring stability and providing regret bounds despite unknown channel statistics.

Contribution

It introduces novel bandit algorithms tailored for channel selection in remote estimation, guaranteeing stability and analyzing regret in the presence of unknown channel parameters.

Findings

Algorithms achieve stable state estimation under unknown channels.

Regret bounds scale sublinearly with time, indicating learning efficiency.

Proposed methods outperform baseline approaches in simulations.

Abstract

In this paper we consider the problem of remote state estimation of a Gauss-Markov process, where a sensor can, at each discrete time instant, transmit on one out of M different communication channels. A key difficulty of the situation at hand is that the channel statistics are unknown. We study the case where both learning of the channel reception probabilities and state estimation is carried out simultaneously. Methods for choosing the channels based on techniques for multi-armed bandits are presented, and shown to provide stability. Furthermore, we define the performance notion of estimation regret, and derive bounds on how it scales with time for the considered algorithms.