Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibria

TL;DR

This paper introduces the concept of active equilibria in multiagent learning, showing they often lead to more effective solutions than traditional Nash equilibria by considering agents' influence on convergence policies.

Contribution

It provides a game-theoretic analysis of active equilibria, demonstrating their advantages over Nash equilibria through comparative examples.

Findings

Active equilibria often outperform Nash equilibria in effectiveness.

Active equilibria account for agents' influence on future policies.

They are a preferred solution concept in multiagent learning.

Abstract

Multiagent learning settings are inherently more difficult than single-agent learning because each agent interacts with other simultaneously learning agents in a shared environment. An effective approach in multiagent reinforcement learning is to consider the learning process of agents and influence their future policies toward desirable behaviors from each agent's perspective. Importantly, if each agent maximizes its long-term rewards by accounting for the impact of its behavior on the set of convergence policies, the resulting multiagent system reaches an active equilibrium. While this new solution concept is general such that standard solution concepts, such as a Nash equilibrium, are special cases of active equilibria, it is unclear when an active equilibrium is a preferred equilibrium over other solution concepts. In this paper, we analyze active equilibria from a game-theoretic perspective by closely studying examples where Nash equilibria are known. By directly comparing active equilibria to Nash equilibria in these examples, we find that active equilibria find more effective solutions than Nash equilibria, concluding that an active equilibrium is the desired solution for multiagent learning settings.