Multi-Threshold AoII-Optimum Sampling Policies for CTMC Information Sources

TL;DR

This paper introduces multi-threshold sampling policies for CTMC sources to minimize the age of incorrect information, formulating the problem as a constrained semi-Markov decision process and proposing practical sub-optimal policies.

Contribution

It develops a novel multi-threshold policy framework for CTMC sources, including a CSMDP formulation and new sub-optimal policies with lower complexity.

Findings

Multi-threshold policies outperform baseline schemes in simulations.

The proposed policies effectively minimize AoII under rate constraints.

The MRPH distribution enables analysis of CTMCs with time-varying transition rates.

Abstract

We study push-based sampling and transmission policies for a status update system consisting of a general finite-state continuous-time Markov chain (CTMC) information source with known dynamics, with the goal of minimizing the average age of incorrect information (AoII). The problem setting we investigate involves an exponentially distributed delay channel for transmissions and a constraint on the average sampling rate. We first show that the optimum sampling and transmission policy is a 'multi-threshold policy', where the thresholds depend on both the estimation value and the state of the original process, and sampling and transmission need to be initiated when the instantaneous AoII exceeds the corresponding threshold, called the estimation- and state-aware transmission (ESAT) policy. Subsequently, we formulate the problem of finding the thresholds as a constrained semi-Markov decision process (CSMDP) and the Lagrangian approach. Additionally, we propose two lower complexity sub-optimum policies, namely the estimation-aware transmission (EAT) policy, and the single-threshold (ST) policy, for which it is possible to obtain these thresholds for CTMCs with relatively larger number of states. The underlying CSMDP formulation relies on the 'multi-regime phase-type' (MRPH) distribution which is a generalization of the well-known phase-type distribution, which allows us to obtain the distribution of time until absorption in a CTMC whose transition rates change with respect to time in a piece-wise manner. The effectiveness of the proposed ESAT, EAT and ST sampling and transmission policies are shown through numerical examples, along with comparisons with a baseline scheme that transmits packets according to a Poisson process in out-of-sync periods.