An active learning method for solving competitive multi-agent decision-making and control problems

TL;DR

This paper presents an active learning approach for identifying stationary action profiles in competitive multi-agent systems by probing agents' reactions and updating local estimates, with theoretical guarantees and numerical validation.

Contribution

It introduces a novel active learning scheme that does not assume the existence of stationary profiles and provides conditions for convergence and existence.

Findings

The method effectively identifies stationary profiles in multi-agent control problems.

Theoretical conditions guarantee asymptotic convergence of the learning scheme.

Numerical simulations demonstrate practical applicability and robustness.

Abstract

To identify a stationary action profile for a population of competitive agents, each executing private strategies, we introduce a novel active-learning scheme where a centralized external observer (or entity) can probe the agents' reactions and recursively update simple local parametric estimates of the action-reaction mappings. Under very general working assumptions (not even assuming that a stationary profile exists), sufficient conditions are established to assess the asymptotic properties of the proposed active learning methodology so that, if the parameters characterizing the action-reaction mappings converge, a stationary action profile is achieved. Such conditions hence act also as certificates for the existence of such a profile. Extensive numerical simulations involving typical competitive multi-agent control and decision-making problems illustrate the practical effectiveness of the proposed learning-based approach.