Mitigating Side Effects in Multi-Agent Systems Using Blame Assignment

TL;DR

This paper presents a decentralized approach for multi-agent systems to mitigate negative side effects by decomposing joint penalties into individual ones, improving scalability and safety in shared environments.

Contribution

It introduces a novel bi-objective lexicographic decentralized MDP model with credit assignment for NSE mitigation in multi-agent systems.

Findings

Effective in reducing negative side effects in simulations

Scalable approach demonstrated with mobile robots

Balances individual objectives with collective safety

Abstract

When independently trained or designed robots are deployed in a shared environment, their combined actions can lead to unintended negative side effects (NSEs). To ensure safe and efficient operation, robots must optimize task performance while minimizing the penalties associated with NSEs, balancing individual objectives with collective impact. We model the problem of mitigating NSEs in a cooperative multi-agent system as a bi-objective lexicographic decentralized Markov decision process. We assume independence of transitions and rewards with respect to the robots' tasks, but the joint NSE penalty creates a form of dependence in this setting. To improve scalability, the joint NSE penalty is decomposed into individual penalties for each robot using credit assignment, which facilitates decentralized policy computation. We empirically demonstrate, using mobile robots and in simulation, the effectiveness and scalability of our approach in mitigating NSEs.