Robust Event-Driven Interactions in Cooperative Multi-Agent Learning

TL;DR

This paper introduces a method for reducing communication in multi-agent systems by using robustness surrogate functions, enabling agents to decide when to communicate based on state deviations, with proven bounds and experimental validation.

Contribution

It proposes a novel distributed decision framework using robustness surrogate functions, including theoretical bounds and data-driven extensions for multi-agent communication efficiency.

Findings

Significant reduction in communication events observed in experiments.

Derived bounds on system optimality based on design parameters.

Extension of methods to learned surrogate functions from data.

Abstract

We present an approach to reduce the communication required between agents in a Multi-Agent learning system by exploiting the inherent robustness of the underlying Markov Decision Process. We compute so-called robustness surrogate functions (off-line), that give agents a conservative indication of how far their state measurements can deviate before they need to update other agents in the system. This results in fully distributed decision functions, enabling agents to decide when it is necessary to update others. We derive bounds on the optimality of the resulting systems in terms of the discounted sum of rewards obtained, and show these bounds are a function of the design parameters. Additionally, we extend the results for the case where the robustness surrogate functions are learned from data, and present experimental results demonstrating a significant reduction in communication events between agents.