Optimizing AoI at Query in Multiuser Wireless Uplink Networks: A Whittle Index Approach

TL;DR

This paper develops a Whittle index-based scheduling policy to optimize the freshness of information in multiuser wireless networks with query arrivals, addressing the complexity of the underlying MDP and demonstrating near-optimal performance.

Contribution

It introduces a novel Whittle index approach for query-based AoI optimization in wireless networks with complex query dynamics, including a new analysis of indexability and an efficient index computation algorithm.

Findings

The proposed policy outperforms baseline scheduling policies.

The analysis confirms the indexability of the sub-MDP with threshold policies.

Simulation results show near-optimal performance of the Whittle index policy.

Abstract

In this paper, we explore how to schedule multiple users to optimize information freshness in a pull-based wireless network, where the status updates from users are requested by randomly arriving queries at the destination. We use the age of information at query (QAoI) to characterize the performance of information freshness. Such a decision-making problem is naturally modeled as a Markov decision process (MDP), which, however, is prohibitively high to be solved optimally by the standard method due to the curse of dimensionality. To address this issue, we employ Whittle index approach, which allows us to decouple the original MDP into multiple sub-MDPs by relaxing the scheduling constraints. However, the binary Markovian query arrival process results in a bi-dimensional state and complex state transitions within each sub-MDP, making it challenging to verify Whittle indexability using conventional methods. After a thorough analysis of the sub-MDP's structure, we show that it is unichain and its optimal policy follows a threshold-type structure. This facilitates the verification of Whittle indexability of the sub-MDP by employing an easy-to-verify condition. Subsequently, the steady-state probability distributions of the sub-MDP under different threshold-type policies are analyzed, constituting the analytical expressions of different Whittle indices in terms of the expected average QAoI and scheduling time of the sub-MDP. Building on these, we devise an efficient algorithm to calculate Whittle indices for the formulated sub-MDPs. The simulation results validate our analyses and show the proposed Whittle index policy outperforms baseline policies and achieves near-optimal performance.