Learning with Limited Shared Information in Multi-agent Multi-armed Bandit

TL;DR

This paper introduces a new multi-agent multi-armed bandit model where agents share limited information, and proposes an algorithm with optimal regret bounds, along with an incentive mechanism to promote participation.

Contribution

The paper presents the LSI-MAMAB model for limited information sharing and the Balanced-ETC algorithm with proven asymptotic optimality and a novel incentive mechanism.

Findings

Balanced-ETC achieves asymptotic optimality.

Average regret approaches a constant with enough agents.

Experimental results validate theoretical analysis.

Abstract

Multi-agent multi-armed bandit (MAMAB) is a classic collaborative learning model and has gained much attention in recent years. However, existing studies do not consider the case where an agent may refuse to share all her information with others, e.g., when some of the data contains personal privacy. In this paper, we propose a novel limited shared information multi-agent multi-armed bandit (LSI-MAMAB) model in which each agent only shares the information that she is willing to share, and propose the Balanced-ETC algorithm to help multiple agents collaborate efficiently with limited shared information. Our analysis shows that Balanced-ETC is asymptotically optimal and its average regret (on each agent) approaches a constant when there are sufficient agents involved. Moreover, to encourage agents to participate in this collaborative learning, an incentive mechanism is proposed to make sure each agent can benefit from the collaboration system. Finally, we present experimental results to validate our theoretical results.