Timely Best Arm Identification in Restless Shared Networks
Mengqiu Zhou, Vincent Y. F. Tan, Meng Zhang

TL;DR
This paper introduces a novel best arm identification approach in restless multi-armed bandits to efficiently select edge nodes with minimal age of information, addressing the challenges of unknown Markovian dynamics.
Contribution
It develops an age-aware LUCB algorithm for restless bandits, providing upper and lower bounds on sample complexity considering Markovian dynamics.
Findings
The proposed algorithm reduces sampling costs compared to benchmarks.
Sample complexity depends on the Markov chain's mixing behavior.
Numerical results validate the effectiveness of the approach under various confidence levels.
Abstract
Real-time status updating applications increasingly rely on networks of devices and edge nodes to maintain data freshness, as quantified by the age of information (AoI) metric. Given that edge computing nodes exhibit uncertain and time-varying dynamics, it is essential to identify the optimal edge node with high confidence and sample efficiency, even without prior knowledge of these dynamics, to ensure timely updates. To address this challenge, we introduce the first best arm identification (BAI) problem aimed at minimizing the long-term average AoI under a fixed confidence setting, framed within the context of a restless multi-armed bandit (RMAB) model. In this model, each arm evolves independently according to an unknown Markov chain over time, regardless of whether it is selected. To capture the temporal trajectories of AoI in the presence of unknown restless dynamics, we develop an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAge of Information Optimization · IoT and Edge/Fog Computing · Opportunistic and Delay-Tolerant Networks
