CaiRL: A High-Performance Reinforcement Learning Environment Toolkit

TL;DR

CaiRL is a high-performance reinforcement learning environment toolkit written in C++, offering faster simulation, legacy Flash game support, and compatibility as a drop-in replacement for OpenAI Gym to enhance training efficiency and sustainability.

Contribution

The paper introduces CaiRL, a C++ based RL toolkit with JVM and Flash support, significantly improving simulation speed and sustainability over existing Python-based tools.

Findings

CaiRL achieves significantly faster environment simulation speeds.

CaiRL can replace OpenAI Gym with minimal modifications.

CaiRL supports legacy Flash games for RL research.

Abstract

This paper addresses the dire need for a platform that efficiently provides a framework for running reinforcement learning (RL) experiments. We propose the CaiRL Environment Toolkit as an efficient, compatible, and more sustainable alternative for training learning agents and propose methods to develop more efficient environment simulations. There is an increasing focus on developing sustainable artificial intelligence. However, little effort has been made to improve the efficiency of running environment simulations. The most popular development toolkit for reinforcement learning, OpenAI Gym, is built using Python, a powerful but slow programming language. We propose a toolkit written in C++ with the same flexibility level but works orders of magnitude faster to make up for Python's inefficiency. This would drastically cut climate emissions. CaiRL also presents the first reinforcement learning toolkit with a built-in JVM and Flash support for running legacy flash games for reinforcement learning research. We demonstrate the effectiveness of CaiRL in the classic control benchmark, comparing the execution speed to OpenAI Gym. Furthermore, we illustrate that CaiRL can act as a drop-in replacement for OpenAI Gym to leverage significantly faster training speeds because of the reduced environment computation time.