CALE: Continuous Arcade Learning Environment

TL;DR

The paper introduces CALE, an extension of the Arcade Learning Environment that supports continuous actions, enabling benchmarking of continuous-control and value-based agents on Atari 2600 games.

Contribution

CALE extends the ALE with continuous actions, allowing for unified evaluation of diverse reinforcement learning agents on Atari environments.

Findings

Initial baseline results with Soft Actor-Critic.

Open research questions and directions provided.

Supports benchmarking of continuous and discrete agents.

Abstract

We introduce the Continuous Arcade Learning Environment (CALE), an extension of the well-known Arcade Learning Environment (ALE) [Bellemare et al., 2013]. The CALE uses the same underlying emulator of the Atari 2600 gaming system (Stella), but adds support for continuous actions. This enables the benchmarking and evaluation of continuous-control agents (such as PPO [Schulman et al., 2017] and SAC [Haarnoja et al., 2018]) and value-based agents (such as DQN [Mnih et al., 2015] and Rainbow [Hessel et al., 2018]) on the same environment suite. We provide a series of open questions and research directions that CALE enables, as well as initial baseline results using Soft Actor-Critic. CALE is available as part of the ALE athttps://github.com/Farama-Foundation/Arcade-Learning-Environment.