Decentralized Contextual Bandits with Network Adaptivity

TL;DR

This paper introduces two network-aware algorithms for decentralized contextual linear bandits, enabling adaptive information sharing that reduces communication costs and improves learning efficiency across networked agents.

Contribution

The paper develops NetLinUCB and Net-SGD-UCB algorithms that decompose learning into global and local parts, allowing efficient, adaptive sharing of information in networked environments with theoretical regret guarantees.

Findings

Regret bounds show sublinear $O(\sqrt{N})$ complexity reduction.

Algorithms outperform benchmarks in simulated pricing tasks.

NetLinUCB and Net-SGD-UCB adapt to different noise and heterogeneity regimes.

Abstract

We consider contextual linear bandits over networks, a class of sequential decision-making problems where learning occurs simultaneously across multiple locations and the reward distributions share structural similarities while also exhibiting local differences. While classical contextual bandits assume either fully centralized data or entirely isolated learners, much remains unexplored in networked environments when information is partially shared. In this paper, we address this gap by developing two network-aware Upper Confidence Bound (UCB) algorithms, NetLinUCB and Net-SGD-UCB, which enable adaptive information sharing guided by dynamically updated network weights. Our approach decompose learning into global and local components and as a result allow agents to benefit from shared structure without full synchronization. Both algorithms incur lighter communication costs compared to a fully centralized setting as agents only share computed summaries regarding the homogeneous features. We establish regret bounds showing that our methods reduce the learning complexity associated with the shared structure from $O(N)$ to sublinear $O(\sqrt{N})$, where $N$ is the size of the network. The two algorithms reveal complementary strengths: NetLinUCB excels in low-noise regimes with fine-grained heterogeneity, while Net-SGD-UCB is robust to high-dimensional, high-variance contexts. We further demonstrate the effectiveness of our methods across simulated pricing environments compared to standard benchmarks.