Unified Distributed Environment

TL;DR

UDE is a versatile toolkit that virtualizes diverse simulation environments for reinforcement learning, enabling multi-agent training across multiple machines with seamless integration into existing RL frameworks.

Contribution

UDE introduces environment virtualization and a unified interface supporting multi-agent RL across various simulation platforms, enhancing flexibility and scalability.

Findings

Supports environments from Gazebo, Unity, Unreal, and OpenAI Gym

Enables remote environment execution with a unified interface

Facilitates multi-agent training across multiple machines

Abstract

We propose Unified Distributed Environment (UDE), an environment virtualization toolkit for reinforcement learning research. UDE is designed to integrate environments built on any simulation platform such as Gazebo, Unity, Unreal, and OpenAI Gym. Through environment virtualization, UDE enables offloading the environment for execution on a remote machine while still maintaining a unified interface. The UDE interface is designed to support multi-agent by default. With environment virtualization and its interface design, the agent policies can be trained in multiple machines for a multi-agent environment. Furthermore, UDE supports integration with existing major RL toolkits for researchers to leverage the benefits. This paper discusses the components of UDE and its design decisions.