Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment

TL;DR

This paper introduces Shbukai, a practical DRL agent for fighting games that improves generalizability, training efficiency, and player alignment, demonstrated in Naruto Mobile with over 100 million users.

Contribution

It presents a novel DRL system with Heterogeneous League Training (HELT) and behavior alignment techniques, tailored for commercial fighting games.

Findings

Shbukai generalizes across all characters despite limited training data.

HELT improves sample efficiency by 22%.

Shbukai enhances player training and engagement.

Abstract

Deep Reinforcement Learning (DRL) agents have demonstrated impressive success in a wide range of game genres. However, existing research primarily focuses on optimizing DRL competence rather than addressing the challenge of prolonged player interaction. In this paper, we propose a practical DRL agent system for fighting games named Sh\=ukai, which has been successfully deployed to Naruto Mobile, a popular fighting game with over 100 million registered users. Sh\=ukai quantifies the state to enhance generalizability, introducing Heterogeneous League Training (HELT) to achieve balanced competence, generalizability, and training efficiency. Furthermore, Sh\=ukai implements specific rewards to align the agent's behavior with human expectations. Sh\=ukai's ability to generalize is demonstrated by its consistent competence across all characters, even though it was trained on only 13% of them. Additionally, HELT exhibits a remarkable 22% improvement in sample efficiency. Sh\=ukai serves as a valuable training partner for players in Naruto Mobile, enabling them to enhance their abilities and skills.