Sample-Efficient Regret-Minimizing Double Oracle in Extensive-Form Games

TL;DR

This paper introduces AdaDO, an adaptive double oracle method that reduces sample complexity from exponential to polynomial in extensive-form games, supported by a new theoretical framework RMDO and empirical validation.

Contribution

It proposes AdaDO with optimal expansion frequency to improve sample efficiency and introduces RMDO for understanding and guiding sample complexity reduction in double oracle algorithms.

Findings

AdaDO achieves lower sample complexity than existing methods.

Empirical results show AdaDO better approximates NE with fewer samples.

Combining RMDO with warm start enhances convergence and scalability.

Abstract

Extensive-Form Game (EFG) represents a fundamental model for analyzing sequential interactions among multiple agents and the primary challenge to solve it lies in mitigating sample complexity. Existing research indicated that Double Oracle (DO) can reduce the sample complexity dependence on the information set number $|S|$ to the final restricted game size $X$ in solving EFG. This is attributed to the early convergence of full-game Nash Equilibrium (NE) through iteratively solving restricted games. However, we prove that the state-of-the-art Extensive-Form Double Oracle (XDO) exhibits \textit{exponential} sample complexity of $X$, due to its exponentially increasing restricted game expansion frequency. Here we introduce Adaptive Double Oracle (AdaDO) to significantly alleviate sample complexity to \textit{polynomial} by deploying the optimal expansion frequency. Furthermore, to comprehensively study the principles and influencing factors underlying sample complexity, we introduce a novel theoretical framework Regret-Minimizing Double Oracle (RMDO) to provide directions for designing efficient DO algorithms. Empirical results demonstrate that AdaDO attains the more superior approximation of NE with less sample complexity than the strong baselines including Linear CFR, MCCFR and existing DO. Importantly, combining RMDO with warm starting and stochastic regret minimization further improves convergence rate and scalability, thereby paving the way for addressing complex multi-agent tasks.