Predicting the expected behavior of agents that learn about agents: the CLRI framework

TL;DR

This paper introduces the CLRI framework, a mathematical model for predicting the behavior of learning agents in multi-agent systems, validated through experiments and theoretical bounds.

Contribution

The paper presents a novel framework and equations for modeling and predicting multi-agent learning dynamics, incorporating parameters like change, learning, and retention rates.

Findings

Validated with reinforcement learning agents in market systems

Provides bounds on learning parameters using PAC-theory

Demonstrates predictive accuracy of the framework

Abstract

We describe a framework and equations used to model and predict the behavior of multi-agent systems (MASs) with learning agents. A difference equation is used for calculating the progression of an agent's error in its decision function, thereby telling us how the agent is expected to fare in the MAS. The equation relies on parameters which capture the agent's learning abilities, such as its change rate, learning rate and retention rate, as well as relevant aspects of the MAS such as the impact that agents have on each other. We validate the framework with experimental results using reinforcement learning agents in a market system, as well as with other experimental results gathered from the AI literature. Finally, we use PAC-theory to show how to calculate bounds on the values of the learning parameters.