Learning Correlated Stackelberg Equilibrium in General-Sum Multi-Leader-Single-Follower Games

TL;DR

This paper introduces a new equilibrium concept called Correlated Stackelberg Equilibrium for multi-leader-single-follower games, along with distributed online learning algorithms that converge to this equilibrium despite noisy feedback.

Contribution

It proposes the novel CSE concept and develops distributed algorithms with regret guarantees for hierarchical multi-player games.

Findings

Algorithms achieve no-external Stackelberg-regret

Convergence to approximate CSE proven

Balances exploration and exploitation in noisy settings

Abstract

Many real-world strategic games involve interactions between multiple players. We study a hierarchical multi-player game structure, where players with asymmetric roles can be separated into leaders and followers, a setting often referred to as Stackelberg game or leader-follower game. In particular, we focus on a Stackelberg game scenario where there are multiple leaders and a single follower, called the Multi-Leader-Single-Follower (MLSF) game. We propose a novel asymmetric equilibrium concept for the MLSF game called Correlated Stackelberg Equilibrium (CSE). We design online learning algorithms that enable the players to interact in a distributed manner, and prove that it can achieve no-external Stackelberg-regret learning. This further translates to the convergence to approximate CSE via a reduction from no-external regret to no-swap regret. At the core of our works, we solve the intricate problem of how to learn equilibrium in leader-follower games with noisy bandit feedback by balancing exploration and exploitation in different learning structures.