Uncertainty-of-Information Scheduling: A Restless Multi-armed Bandit Framework

TL;DR

This paper introduces a novel scheduling framework based on the uncertainty of information (UoI) for binary Markov processes, formulating it as a restless multi-armed bandit problem and proposing a near-optimal Whittle index policy.

Contribution

It develops a Whittle index policy for UoI-based scheduling in RMABs, proving its indexability and providing an iterative algorithm for practical implementation.

Findings

Whittle index policy achieves near-optimal performance.

The policy is applicable to a broad class of RMABs with binary Markov processes.

Numerical results confirm the effectiveness of the proposed approach.

Abstract

This paper proposes using the uncertainty of information (UoI), measured by Shannon's entropy, as a metric for information freshness. We consider a system in which a central monitor observes multiple binary Markov processes through a communication channel. The UoI of a Markov process corresponds to the monitor's uncertainty about its state. At each time step, only one Markov process can be selected to update its state to the monitor; hence there is a tradeoff among the UoIs of the processes that depend on the scheduling policy used to select the process to be updated. The age of information (AoI) of a process corresponds to the time since its last update. In general, the associated UoI can be a non-increasing function, or even an oscillating function, of its AoI, making the scheduling problem particularly challenging. This paper investigates scheduling policies that aim to minimize the average sum-UoI of the processes over the infinite time horizon. We formulate the problem as a restless multi-armed bandit (RMAB) problem, and develop a Whittle index policy that is near-optimal for the RMAB after proving its indexability. We further provide an iterative algorithm to compute the Whittle index for the practical deployment of the policy. Although this paper focuses on UoI scheduling, our results apply to a general class of RMABs for which the UoI scheduling problem is a special case. Specifically, this paper's Whittle index policy is valid for any RMAB in which the bandits are binary Markov processes and the penalty is a concave function of the belief state of the Markov process. Numerical results demonstrate the excellent performance of the Whittle index policy for this class of RMABs.