Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Play

TL;DR

This paper introduces an end-to-end policy and an optimistic smooth fictitious play algorithm to master a multi-stage strategy card game, achieving state-of-the-art results and winning championships.

Contribution

It presents a novel approach combining deep reinforcement learning with fictitious play tailored for multi-stage games with varying observation and action spaces.

Findings

Wins double championships at COG2022

Outperforms existing methods on Legends of Code and Magic

Demonstrates effectiveness of the proposed algorithms in complex multi-stage settings

Abstract

Deep Reinforcement Learning combined with Fictitious Play shows impressive results on many benchmark games, most of which are, however, single-stage. In contrast, real-world decision making problems may consist of multiple stages, where the observation spaces and the action spaces can be completely different across stages. We study a two-stage strategy card game Legends of Code and Magic and propose an end-to-end policy to address the difficulties that arise in multi-stage game. We also propose an optimistic smooth fictitious play algorithm to find the Nash Equilibrium for the two-player game. Our approach wins double championships of COG2022 competition. Extensive studies verify and show the advancement of our approach.