Learning Sharing Behaviors with Arbitrary Numbers of Agents

TL;DR

This paper introduces a method for modeling turn-taking behaviors among multiple agents using WFSTs and a logistic regression fusion model, enabling accurate behavior prediction and effective turn-taking in shared-resource environments.

Contribution

It presents a novel multi-agent behavior modeling approach that is independent of the number of agents, combining WFSTs with a fusion model for accurate behavior prediction.

Findings

Model achieves 0.63 to 1.0 accuracy in behavior prediction.

KL-divergence less than 0.1 with single agent, less than 0.37 with multiple agents.

Q-learning agent successfully uses the model for turn-taking.

Abstract

We propose a method for modeling and learning turn-taking behaviors for accessing a shared resource. We model the individual behavior for each agent in an interaction and then use a multi-agent fusion model to generate a summary over the expected actions of the group to render the model independent of the number of agents. The individual behavior models are weighted finite state transducers (WFSTs) with weights dynamically updated during interactions, and the multi-agent fusion model is a logistic regression classifier. We test our models in a multi-agent tower-building environment, where a Q-learning agent learns to interact with rule-based agents. Our approach accurately models the underlying behavior patterns of the rule-based agents with accuracy ranging between 0.63 and 1.0 depending on the stochasticity of the other agent behaviors. In addition we show using KL-divergence that the model accurately captures the distribution of next actions when interacting with both a single agent (KL-divergence < 0.1) and with multiple agents (KL-divergence < 0.37). Finally, we demonstrate that our behavior model can be used by a Q-learning agent to take turns in an interactive turn-taking environment.