# Google Research Football: A Novel Reinforcement Learning Environment

## TL;DR

The paper introduces Google Research Football, a new physics-based 3D reinforcement learning environment for training agents in complex football scenarios, supporting multiplayer and multi-agent experiments.

## Contribution

It presents a customizable, open-source football simulation environment with benchmark scenarios and baseline results for popular RL algorithms, enabling advanced research and testing.

## Key findings

- Baseline results for IMPALA, PPO, and Ape-X DQN algorithms.
- Three full-game scenarios with varying difficulty levels.
- A diverse set of simpler scenarios for research exploration.

## Abstract

Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly tested in a safe and reproducible manner. We introduce the Google Research Football Environment, a new reinforcement learning environment where agents are trained to play football in an advanced, physics-based 3D simulator. The resulting environment is challenging, easy to use and customize, and it is available under a permissive open-source license. In addition, it provides support for multiplayer and multi-agent experiments. We propose three full-game scenarios of varying difficulty with the Football Benchmarks and report baseline results for three commonly used reinforcement algorithms (IMPALA, PPO, and Ape-X DQN). We also provide a diverse set of simpler scenarios with the Football Academy and showcase several promising research directions.