A Survey of Deep Reinforcement Learning in Video Games

TL;DR

This survey reviews the progress and challenges of deep reinforcement learning in video game AI, covering various algorithms, achievements, and future research directions in the field.

Contribution

It provides a comprehensive overview of DRL methods applied to video games, comparing techniques and highlighting key challenges and research opportunities.

Findings

DRL has achieved super-human performance in many video games.

Different DRL algorithms vary in techniques and properties.

Challenges include exploration, sample efficiency, and multi-agent learning.

Abstract

Deep reinforcement learning (DRL) has made great achievements since proposed. Generally, DRL agents receive high-dimensional inputs at each step, and make actions according to deep-neural-network-based policies. This learning mechanism updates the policy to maximize the return with an end-to-end method. In this paper, we survey the progress of DRL methods, including value-based, policy gradient, and model-based algorithms, and compare their main techniques and properties. Besides, DRL plays an important role in game artificial intelligence (AI). We also take a review of the achievements of DRL in various video games, including classical Arcade games, first-person perspective games and multi-agent real-time strategy games, from 2D to 3D, and from single-agent to multi-agent. A large number of video game AIs with DRL have achieved super-human performance, while there are still some challenges in this domain. Therefore, we also discuss some key points when applying DRL methods to this field, including exploration-exploitation, sample efficiency, generalization and transfer, multi-agent learning, imperfect information, and delayed spare rewards, as well as some research directions.