Dynamic Multi-Arm Bandit Game Based Multi-Agents Spectrum Sharing Strategy Design

TL;DR

This paper formulates a multi-agent spectrum sharing problem in wireless avionics using multi-arm bandit game theory, applying UCB and Thompson Sampling algorithms to optimize spectrum utilization and minimize regret.

Contribution

It introduces a novel multi-agent spectrum sharing strategy based on multi-arm bandit game theory, including both UCB and Markov game frameworks, with comprehensive performance evaluation.

Findings

UCB achieves near-optimal reward maximization

Thompson Sampling serves as an effective benchmark

Optimal parameters for UCB are identified

Abstract

For a wireless avionics communication system, a Multi-arm bandit game is mathematically formulated, which includes channel states, strategies, and rewards. The simple case includes only two agents sharing the spectrum which is fully studied in terms of maximizing the cumulative reward over a finite time horizon. An Upper Confidence Bound (UCB) algorithm is used to achieve the optimal solutions for the stochastic Multi-Arm Bandit (MAB) problem. Also, the MAB problem can also be solved from the Markov game framework perspective. Meanwhile, Thompson Sampling (TS) is also used as benchmark to evaluate the proposed approach performance. Numerical results are also provided regarding minimizing the expectation of the regret and choosing the best parameter for the upper confidence bound.