On Partially Controlled Multi-Agent Systems

TL;DR

This paper explores how to influence uncontrollable agents in multi-agent systems by designing controllable agents, using game theory to analyze different control methods across utility maximizers and reinforcement learners.

Contribution

It introduces a framework for partially controlled multi-agent systems and compares control techniques for different types of uncontrollable agents.

Findings

Control methods vary in effectiveness between utility maximizers and reinforcement learners.

Game-theoretic analysis reveals relationships between control strategies across domains.

Design principles for influencing uncontrollable agents are identified.

Abstract

Motivated by the control theoretic distinction between controllable and uncontrollable events, we distinguish between two types of agents within a multi-agent system: controllable agents, which are directly controlled by the system's designer, and uncontrollable agents, which are not under the designer's direct control. We refer to such systems as partially controlled multi-agent systems, and we investigate how one might influence the behavior of the uncontrolled agents through appropriate design of the controlled agents. In particular, we wish to understand which problems are naturally described in these terms, what methods can be applied to influence the uncontrollable agents, the effectiveness of such methods, and whether similar methods work across different domains. Using a game-theoretic framework, this paper studies the design of partially controlled multi-agent systems in two contexts: in one context, the uncontrollable agents are expected utility maximizers, while in the other they are reinforcement learners. We suggest different techniques for controlling agents' behavior in each domain, assess their success, and examine their relationship.