Deep Reinforcement Learning for General Video Game AI

TL;DR

This paper bridges the GVGAI framework with OpenAI Gym to evaluate deep reinforcement learning algorithms across multiple video games, providing insights into game difficulty and algorithm performance.

Contribution

It introduces an interface connecting GVGAI to OpenAI Gym and evaluates deep RL algorithms on various GVGAI games for the first time.

Findings

Deep RL algorithms vary in performance across GVGAI games.

Some games are significantly more challenging for RL agents.

The study offers a benchmark for future RL research on video games.

Abstract

The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.