Federated Online Sparse Decision Making

TL;DR

This paper introduces Fedego Lasso, a federated bandit algorithm that leverages sparsity and collaborative learning to efficiently handle heterogeneous clients with high-dimensional decision contexts, achieving near-optimal regret with minimal communication.

Contribution

It proposes a novel federated linear contextual bandit model with a sparse reward structure and introduces Fedego Lasso, a collaborative algorithm that manages heterogeneity without sharing raw data.

Findings

Achieves near-optimal regret bounds with logarithmic communication.

Effectively handles heterogeneity across clients in high-dimensional settings.

Demonstrates superior performance on synthetic and real-world datasets.

Abstract

This paper presents a novel federated linear contextual bandits model, where individual clients face different K-armed stochastic bandits with high-dimensional decision context and coupled through common global parameters. By leveraging the sparsity structure of the linear reward , a collaborative algorithm named \texttt{Fedego Lasso} is proposed to cope with the heterogeneity across clients without exchanging local decision context vectors or raw reward data. \texttt{Fedego Lasso} relies on a novel multi-client teamwork-selfish bandit policy design, and achieves near-optimal regrets for shared parameter cases with logarithmic communication costs. In addition, a new conceptual tool called federated-egocentric policies is introduced to delineate exploration-exploitation trade-off. Experiments demonstrate the effectiveness of the proposed algorithms on both synthetic and real-world datasets.