Distributed Learning in Markovian Restless Bandits over Interference Graphs for Stable Spectrum Sharing

TL;DR

This paper introduces SMILE, a distributed learning algorithm for spectrum sharing in wireless networks modeled by interference graphs, achieving stable, interference-aware channel allocation with proven convergence and low regret.

Contribution

It is the first to establish global Gale-Shapley stability in a stochastic restless environment and develops a communication-efficient algorithm combining restless bandit learning with graph coordination.

Findings

SMILE converges to the optimal stable allocation.

SMILE achieves logarithmic regret compared to full-knowledge benchmarks.

Simulations confirm robustness, scalability, and efficiency of SMILE.

Abstract

We study distributed learning for spectrum access and sharing among multiple cognitive communication entities, such as cells, subnetworks, or cognitive radio users (collectively referred to as cells), in communication-constrained wireless networks modeled by interference graphs. Our goal is to achieve a globally stable and interference-aware channel allocation. Stability is defined through a generalized Gale-Shapley multi-to-one matching, a well-established solution concept in wireless resource allocation. We consider wireless networks where L cells share S orthogonal channels and cannot simultaneously use the same channel as their neighbors. Each channel evolves as an unknown restless Markov process with cell-dependent rewards, making this the first work to establish global Gale-Shapley stability for channel allocation in a stochastic, temporally varying restless environment. To address this challenge, we develop SMILE (Stable Multi-matching with Interference-aware LEarning), a communication-efficient distributed learning algorithm that integrates restless bandit learning with graph-constrained coordination. SMILE enables cells to distributedly balance exploration of unknown channels with exploitation of learned information. We prove that SMILE converges to the optimal stable allocation and achieves logarithmic regret relative to a genie with full knowledge of expected utilities. Simulations validate the theoretical guarantees and demonstrate SMILE's robustness, scalability, and efficiency across diverse spectrum-sharing scenarios.