Sim-Env: Decoupling OpenAI Gym Environments from Simulation Models

TL;DR

Sim-Env is a Python library that simplifies the creation and swapping of OpenAI Gym environments by decoupling simulation models from environment interfaces, enhancing flexibility and modularity in reinforcement learning research.

Contribution

It introduces a workflow and tools for decoupled development of RL environments, allowing easy swapping of simulation back-ends without altering domain models.

Findings

Enables seamless environment swapping in RL workflows

Supports reuse of domain models across multiple environments

Enhances modularity and ease of maintenance in RL development

Abstract

Reinforcement learning (RL) is one of the most active fields of AI research. Despite the interest demonstrated by the research community in reinforcement learning, the development methodology still lags behind, with a severe lack of standard APIs to foster the development of RL applications. OpenAI Gym is probably the most used environment to develop RL applications and simulations, but most of the abstractions proposed in such a framework are still assuming a semi-structured methodology. This is particularly relevant for agent-based models whose purpose is to analyse adaptive behaviour displayed by self-learning agents in the simulation. In order to bridge this gap, we present a workflow and tools for the decoupled development and maintenance of multi-purpose agent-based models and derived single-purpose reinforcement learning environments, enabling the researcher to swap out environments with ones representing different perspectives or different reward models, all while keeping the underlying domain model intact and separate. The Sim-Env Python library generates OpenAI-Gym-compatible reinforcement learning environments that use existing or purposely created domain models as their simulation back-ends. Its design emphasizes ease-of-use, modularity and code separation.