Semi-overlapping Multi-bandit Best Arm Identification for Sequential Support Network Learning

TL;DR

This paper introduces a new semi-overlapping multi-bandit model and a generalized algorithm for efficiently identifying optimal candidates in complex, shared evaluation settings, with applications in various multi-agent learning scenarios.

Contribution

It proposes the semi-overlapping multi-bandit model and develops a generalized GapE algorithm with improved error bounds for support network learning.

Findings

Exponential error bounds scale linearly with overlap degree.

Shared evaluations significantly reduce sample complexity.

Theoretical guarantees improve upon existing multi-bandit best-arm identification methods.

Abstract

Many modern AI and ML problems require evaluating partners' contributions through shared yet asymmetric, computationally intensive processes and the simultaneous selection of the most beneficial candidates. Sequential approaches to these problems can be unified under a new framework, Sequential Support Network Learning (SSNL), in which the goal is to select the most beneficial candidate set of partners for all participants using trials; that is, to learn a directed graph that represents the highest-performing contributions. We demonstrate that a new pure-exploration model, the semi-overlapping multi-(multi-armed) bandit (SOMMAB), in which a single evaluation provides distinct feedback to multiple bandits due to structural overlap among their arms, can be used to learn a support network from sparse candidate lists efficiently. We develop a generalized GapE algorithm for SOMMABs and derive new exponential error bounds that improve the best known constant in the exponent for multi-bandit best-arm identification. The bounds scale linearly with the degree of overlap, revealing significant sample-complexity gains arising from shared evaluations. From an application point of view, this work provides a theoretical foundation and improved performance guarantees for sequential learning tools for identifying support networks from sparse candidates in multiple learning problems, such as in multi-task learning (MTL), auxiliary task learning (ATL), federated learning (FL), and in multi-agent systems (MAS).