Last-Iterate Convergence in Adaptive Regret Minimization for Approximate Extensive-Form Perfect Equilibrium

TL;DR

This paper introduces an adaptive regret minimization algorithm that efficiently computes approximate extensive-form perfect equilibria with last-iterate convergence, improving over existing methods in two-player zero-sum extensive-form games.

Contribution

It presents a novel adaptive regret minimization approach with last-iterate convergence for approximate EFPE, addressing computational and approximation limitations of prior algorithms.

Findings

The proposed method achieves last-iterate convergence to EFPE.

Experimental results outperform state-of-the-art algorithms in NE and EFPE tasks.

The algorithm effectively balances convergence speed and approximation accuracy.

Abstract

The Nash Equilibrium (NE) assumes rational play in imperfect-information Extensive-Form Games (EFGs) but fails to ensure optimal strategies for off-equilibrium branches of the game tree, potentially leading to suboptimal outcomes in practical settings. To address this, the Extensive-Form Perfect Equilibrium (EFPE), a refinement of NE, introduces controlled perturbations to model potential player errors. However, existing EFPE-finding algorithms, which typically rely on average strategy convergence and fixed perturbations, face significant limitations: computing average strategies incurs high computational costs and approximation errors, while fixed perturbations create a trade-off between NE approximation accuracy and the convergence rate of NE refinements. To tackle these challenges, we propose an efficient adaptive regret minimization algorithm for computing approximate EFPE, achieving last-iterate convergence in two-player zero-sum EFGs. Our approach introduces Reward Transformation Counterfactual Regret Minimization (RTCFR) to solve perturbed games and defines a novel metric, the Information Set Nash Equilibrium (ISNE), to dynamically adjust perturbations. Theoretical analysis confirms convergence to EFPE, and experimental results demonstrate that our method significantly outperforms state-of-the-art algorithms in both NE and EFPE-finding tasks.