Learning to Coordinate Under Threshold Rewards: A Cooperative Multi-Agent Bandit Framework

TL;DR

This paper introduces a decentralized multi-agent bandit algorithm that learns activation thresholds and avoids decoys, enabling effective coordination under threshold-based rewards and outperforming baseline methods.

Contribution

The paper presents T-Coop-UCB, a novel algorithm for decentralized learning of thresholds and rewards in multi-agent bandit problems with decoys, advancing coordination capabilities.

Findings

T-Coop-UCB outperforms baseline methods in cumulative reward.

The algorithm achieves low regret and effective coordination.

Near-Oracle performance demonstrates its efficiency.

Abstract

Cooperative multi-agent systems often face tasks that require coordinated actions under uncertainty. While multi-armed bandit (MAB) problems provide a powerful framework for decentralized learning, most prior work assumes individually attainable rewards. We address the challenging setting where rewards are threshold-activated: an arm yields a payoff only when a minimum number of agents pull it simultaneously, with this threshold unknown in advance. Complicating matters further, some arms are decoys - requiring coordination to activate but yielding no reward - introducing a new challenge of wasted joint exploration. We introduce Threshold-Coop-UCB (T-Coop-UCB), a decentralized algorithm that enables agents to jointly learn activation thresholds and reward distributions, forming effective coalitions without centralized control. Empirical results show that T-Coop-UCB consistently outperforms baseline methods in cumulative reward, regret, and coordination metrics, achieving near-Oracle performance. Our findings underscore the importance of joint threshold learning and decoy avoidance for scalable, decentralized cooperation in complex multi-agent