The Signaler-Responder Game: Learning to Communicate using Thompson Sampling

TL;DR

This paper introduces a distributed Bayesian learning approach using Thompson Sampling for agents in a signaler-responder game to learn effective communication and cooperation strategies without pre-programmed rules.

Contribution

It presents two novel Thompson Sampling-based algorithms enabling heterogeneous agents to learn optimal communication strategies in a multi-agent game setting.

Findings

Agents adapt strategies over time to maximize rewards.

Communication occurs only when beneficial, reducing unnecessary costs.

The algorithms converge to efficient equilibria under various conditions.

Abstract

We are interested in studying how heterogeneous agents can learn to communicate and cooperate with each other without being explicitly pre-programmed to do so. Motivated by this goal, we present and analyze a distributed solution to a two-player signaler-responder game which is defined as follows. The signaler agent has a random, exogenous need and can choose from four different strategies: never signal, always signal, signal when need, and signal when no need. The responder agent can choose to either ignore or respond to the signal. We define a reward to both agents when they cooperate to satisfy the signaler's need, and costs associated with communication, response and unmet needs. We identify pure Nash equilibria of the game and the conditions under which they occur. As a solution for this game, we propose two new distributed Bayesian learning algorithms, one for each agent, based on the classic Thompson Sampling policy for multi-armed bandits. These algorithms allow both agents to update beliefs about both the exogenous need and the behavior of the other agent and optimize their own expected reward. We show that by using these policies, the agents are able to intelligently adapt their strategies over multiple iterations to attain efficient, reward-maximizing equilibria under different settings, communicating and cooperating when it is rewarding to do so, and not communicating or cooperating when it is too expensive.