Learning to Beat ByteRL: Exploitability of Collectible Card Game Agents

TL;DR

This paper analyzes the exploitability of ByteRL, a state-of-the-art collectible card game agent, revealing its strengths in Hearthstone but vulnerabilities in Legends of Code and Magic due to the game's complexity.

Contribution

It provides the first analysis of ByteRL's exploitability in different collectible card games, highlighting the challenges and techniques for AI in large state space environments.

Findings

ByteRL beats a top-10 Hearthstone player.

ByteRL is highly exploitable in Legends of Code and Magic.

Challenges in applying search methods to large state spaces.

Abstract

While Poker, as a family of games, has been studied extensively in the last decades, collectible card games have seen relatively little attention. Only recently have we seen an agent that can compete with professional human players in Hearthstone, one of the most popular collectible card games. Although artificial agents must be able to work with imperfect information in both of these genres, collectible card games pose another set of distinct challenges. Unlike in many poker variants, agents must deal with state space so vast that even enumerating all states consistent with the agent's beliefs is intractable, rendering the current search methods unusable and requiring the agents to opt for other techniques. In this paper, we investigate the strength of such techniques for this class of games. Namely, we present preliminary analysis results of ByteRL, the state-of-the-art agent in Legends of Code and Magic and Hearthstone. Although ByteRL beat a top-10 Hearthstone player from China, we show that its play in Legends of Code and Magic is highly exploitable.