Distributed Learning in the Presence of Disturbances

TL;DR

This paper extends a distributed learning algorithm to handle disturbances, enabling agents to learn Pareto-efficient solutions in complex systems with communication constraints and unknown utilities.

Contribution

It introduces a modified distributed learning algorithm that accounts for disturbances, broadening its applicability to more realistic, dynamic environments.

Findings

Algorithm successfully learns Pareto-efficient solutions despite disturbances.

Application to traffic ramp coordination demonstrates practical effectiveness.

Approach handles finite set of disturbances in distributed settings.

Abstract

We consider a problem where multiple agents must learn an action profile that maximises the sum of their utilities in a distributed manner. The agents are assumed to have no knowledge of either the utility functions or the actions and payoffs of other agents. These assumptions arise when modelling the interactions in a complex system and communicating between various components of the system are both difficult. In [1], a distributed algorithm was proposed, which learnt Pareto-efficient solutions in this problem setting. However, the approach assumes that all agents can choose their actions, which precludes disturbances. In this paper, we show that a modified version of this distributed learning algorithm can learn Pareto-efficient solutions, even in the presence of disturbances from a finite set. We apply our approach to the problem of ramp coordination in traffic control for different demand profiles.