In-Context Exploiter for Extensive-Form Games

TL;DR

This paper introduces In-Context Exploiter (ICE), a novel transformer-based method that learns to exploit any opponent in extensive-form games through in-context learning, surpassing traditional Nash equilibrium strategies.

Contribution

We propose ICE, a pioneering in-context learning approach enabling a single model to adaptively exploit diverse opponents in extensive-form games.

Findings

ICE effectively exploits unknown opponents in various game scenarios.

The method outperforms traditional equilibrium-based strategies.

Experimental results demonstrate ICE's adaptability and robustness.

Abstract

Nash equilibrium (NE) is a widely adopted solution concept in game theory due to its stability property. However, we observe that the NE strategy might not always yield the best results, especially against opponents who do not adhere to NE strategies. Based on this observation, we pose a new game-solving question: Can we learn a model that can exploit any, even NE, opponent to maximize their own utility? In this work, we make the first attempt to investigate this problem through in-context learning. Specifically, we introduce a novel method, In-Context Exploiter (ICE), to train a single model that can act as any player in the game and adaptively exploit opponents entirely by in-context learning. Our ICE algorithm involves generating diverse opponent strategies, collecting interactive history training data by a reinforcement learning algorithm, and training a transformer-based agent within a well-designed curriculum learning framework. Finally, comprehensive experimental results validate the effectiveness of our ICE algorithm, showcasing its in-context learning ability to exploit any unknown opponent, thereby positively answering our initial game-solving question.