CoRL: Environment Creation and Management Focused on System Integration

TL;DR

CoRL is a flexible, modular environment creation library for reinforcement learning that offers detailed control over environment parameters, supports multi-agent systems, and facilitates rapid integration with various simulation environments.

Contribution

It introduces a highly configurable, composable environment creation framework that improves flexibility and scalability over existing RL environment libraries.

Findings

Enables detailed customization of agent observations and rewards.

Supports multi-agent environments with scalable integration.

Facilitates rapid transition from low- to high-fidelity simulations.

Abstract

Existing reinforcement learning environment libraries use monolithic environment classes, provide shallow methods for altering agent observation and action spaces, and/or are tied to a specific simulation environment. The Core Reinforcement Learning library (CoRL) is a modular, composable, and hyper-configurable environment creation tool. It allows minute control over agent observations, rewards, and done conditions through the use of easy-to-read configuration files, pydantic validators, and a functor design pattern. Using integration pathways allows agents to be quickly implemented in new simulation environments, encourages rapid exploration, and enables transition of knowledge from low-fidelity to high-fidelity simulations. Natively multi-agent design and integration with Ray/RLLib (Liang et al., 2018) at release allow for easy scalability of agent complexity and computing power. The code is publicly released and available at https://github.com/act3-ace/CoRL.