Safe Opponent Exploitation For Epsilon Equilibrium Strategies

TL;DR

This paper introduces prime-safe opponent exploitation, a method that enhances safe exploitation strategies in large imperfect information games by redefining game value to ensure practical guarantees and improved empirical performance.

Contribution

It extends safe opponent exploitation to large domains by redefining game value as a worst-case payoff, enabling scalable and practical exploitation strategies.

Findings

Empirical improvements over existing safe exploitation algorithms.

Effective in large imperfect information games like poker.

Provides practical guarantees with weaker epsilon strategies.

Abstract

In safe opponent exploitation players hope to exploit their opponents' potentially sub-optimal strategies while guaranteeing at least the value of the game in expectation for themselves. Safe opponent exploitation algorithms have been successfully applied to small instances of two-player zero-sum imperfect information games, where Nash equilibrium strategies are typically known in advance. Current methods available to compute these strategies are however not scalable to desirable large domains of imperfect information such as No-Limit Texas Hold 'em (NLHE) poker, where successful agents rely on game abstractions in order to compute an equilibrium strategy approximation. This paper will extend the concept of safe opponent exploitation by introducing prime-safe opponent exploitation, in which we redefine the value of the game of a player to be the worst-case payoff their strategy could be susceptible to. This allows weaker epsilon equilibrium strategies to benefit from utilising a form of opponent exploitation with our revised value of the game, still allowing for a practical game-theoretical guaranteed lower-bound. We demonstrate the empirical advantages of our generalisation when applied to the main safe opponent exploitation algorithms.