Mirror Mode in Fire Emblem: Beating Players at their own Game with Imitation and Reinforcement Learning

TL;DR

This paper introduces Mirror Mode in a turn-based game where AI mimics player strategies using reinforcement and imitation learning, enhancing challenge and player engagement.

Contribution

It presents a novel game mode that employs combined imitation and reinforcement learning to imitate player strategies, increasing game unpredictability and player satisfaction.

Findings

Good imitation of defensive strategies achieved

Players recognized their own retreat tactics

Mirror Mode increased player satisfaction

Abstract

Enemy strategies in turn-based games should be surprising and unpredictable. This study introduces Mirror Mode, a new game mode where the enemy AI mimics the personal strategy of a player to challenge them to keep changing their gameplay. A simplified version of the Nintendo strategy video game Fire Emblem Heroes has been built in Unity, with a Standard Mode and a Mirror Mode. Our first set of experiments find a suitable model for the task to imitate player demonstrations, using Reinforcement Learning and Imitation Learning: combining Generative Adversarial Imitation Learning, Behavioral Cloning, and Proximal Policy Optimization. The second set of experiments evaluates the constructed model with player tests, where models are trained on demonstrations provided by participants. The gameplay of the participants indicates good imitation in defensive behavior, but not in offensive strategies. Participant's surveys indicated that they recognized their own retreating tactics, and resulted in an overall higher player-satisfaction for Mirror Mode. Refining the model further may improve imitation quality and increase player's satisfaction, especially when players face their own strategies. The full code and survey results are stored at: https://github.com/YannaSmid/MirrorMode