Near-Optimal Privacy-Preserving Learning for Max-Min Fair Multi-Agent Bandits

TL;DR

This paper introduces a distributed, privacy-preserving algorithm for max-min fair multi-agent bandit learning that achieves near-optimal regret with polynomial dependence on the number of agents and near-logarithmic dependence on the time horizon.

Contribution

It proposes a novel fully distributed algorithm that preserves reward privacy and improves upon previous methods by reducing the dependence on the number of agents from exponential to polynomial.

Findings

Achieves regret $O(N^3 f( ext{log } T) ext{log } T)$ with unknown reward support.

Maintains reward privacy by avoiding reward sharing among agents.

Simulation results confirm the theoretical scaling with horizon, agents, and gap.

Abstract

We study fair multi-agent multi-armed bandit learning under collision-only coordination. Agents cannot communicate explicitly during learning and observe only their own rewards and whether collisions occur when several agents access the same arm. The goal is to learn a max-min fair allocation while keeping each agent's reward samples and empirical reward estimates local. We propose a fully distributed algorithm for bounded rewards with unknown support, achieving regret $O\!\left(N^3 f(\log T)\log T\right)$, where $f$ is any nondecreasing diverging function satisfying $f(k-1)/f(k)\to 1$. The algorithm combines distributed agent ordering, cumulative round-robin exploration, endpoint-revalidated warm-started bisection, and a collision-based distributed auction for threshold-feasibility tests. Unlike leader-based optimal algorithms, no agent collects the reward observations, empirical estimates, or preferences of the others. Thus, the protocol preserves reward privacy in the operational sense of avoiding reward sharing, while coordinating only through collision outcomes. Compared with previous privacy-preserving algorithms for max--min fair bandits, which have exponential dependence on the number of agents, our method achieves polynomial $N^3$ dependence while retaining near-logarithmic dependence on $T$. The analysis uses concentration of cumulative empirical estimates and stability of endpoint-revalidated bisection. Simulations confirm the predicted scaling with horizon, number of agents, and max--min gap across representative numerical settings.