Centralized active tracking of a Markov chain with unknown dynamics

TL;DR

This paper proposes an online algorithm for active sensor selection to track a Markov chain with unknown dynamics, balancing estimation accuracy and energy efficiency, using Gibbs sampling and EM techniques.

Contribution

It introduces a novel online method combining Gibbs sampling and EM to estimate unknown Markov chain parameters while optimizing sensor activation under constraints.

Findings

Achieves about 1 dB better MSE than uniform sampling.

Maintains within 2 dB of full observation performance.

Demonstrates practical implementability of the proposed algorithm.

Abstract

In this paper, selection of an active sensor subset for tracking a discrete time, finite state Markov chain having an unknown transition probability matrix (TPM) is considered. A total of N sensors are available for making observations of the Markov chain, out of which a subset of sensors are activated each time in order to perform reliable estimation of the process. The trade-off is between activating more sensors to gather more observations for the remote estimation, and restricting sensor usage in order to save energy and bandwidth consumption. The problem is formulated as a constrained minimization problem, where the objective is the long-run averaged mean-squared error (MSE) in estimation, and the constraint is on sensor activation rate. A Lagrangian relaxation of the problem is solved by an artful blending of two tools: Gibbs sampling for MSE minimization and an on-line version of expectation maximization (EM) to estimate the unknown TPM. Finally, the Lagrange multiplier is updated using slower timescale stochastic approximation in order to satisfy the sensor activation rate constraint. The on-line EM algorithm, though adapted from literature, can estimate vector-valued parameters even under time-varying dimension of the sensor observations. Numerical results demonstrate approximately 1 dB better error performance than uniform sensor sampling and comparable error performance (within 2 dB bound) against complete sensor observation. This makes the proposed algorithm amenable to practical implementation.