Safe Subgame Resolving for Extensive Form Correlated Equilibrium

TL;DR

This paper introduces subgame resolving techniques for extensive form correlated equilibrium (EFCE), enabling efficient, online strategy refinement in general-sum games with guarantees of safety and improved computational feasibility.

Contribution

It develops the first online subgame resolving algorithms for EFCE, providing theoretical foundations and practical methods with safety guarantees for general-sum extensive form games.

Findings

Subgame resolving can be applied efficiently to EFCEs.

The proposed algorithms guarantee safety and never harm social welfare.

EFCEs have sufficient independence between subgames for resolving to be effective.

Abstract

Correlated Equilibrium is a solution concept that is more general than Nash Equilibrium (NE) and can lead to outcomes with better social welfare. However, its natural extension to the sequential setting, the \textit{Extensive Form Correlated Equilibrium} (EFCE), requires a quadratic amount of space to solve, even in restricted settings without randomness in nature. To alleviate these concerns, we apply \textit{subgame resolving}, a technique extremely successful in finding NE in zero-sum games to solving general-sum EFCEs. Subgame resolving refines a correlation plan in an \textit{online} manner: instead of solving for the full game upfront, it only solves for strategies in subgames that are reached in actual play, resulting in significant computational gains. In this paper, we (i) lay out the foundations to quantify the quality of a refined strategy, in terms of the \textit{social welfare} and \textit{exploitability} of correlation plans, (ii) show that EFCEs possess a sufficient amount of independence between subgames to perform resolving efficiently, and (iii) provide two algorithms for resolving, one using linear programming and the other based on regret minimization. Both methods guarantee \textit{safety}, i.e., they will never be counterproductive. Our methods are the first time an online method has been applied to the correlated, general-sum setting.