Karolos: An Open-Source Reinforcement Learning Framework for Robot-Task Environments

TL;DR

Karolos is an open-source reinforcement learning framework tailored for robotic applications, emphasizing modularity, speed, and flexibility to facilitate research and real-world deployment of RL algorithms in robot-task transfer scenarios.

Contribution

The paper introduces Karolos, a flexible, efficient RL framework for robotics that supports modular environments, state-of-the-art algorithms, and parallelization to accelerate experiments.

Findings

Supports transfer learning in robotics

Enables faster experimentation through parallel environments

Includes implementations of leading RL algorithms

Abstract

In reinforcement learning (RL) research, simulations enable benchmarks between algorithms, as well as prototyping and hyper-parameter tuning of agents. In order to promote RL both in research and real-world applications, frameworks are required which are on the one hand efficient in terms of running experiments as fast as possible. On the other hand, they must be flexible enough to allow the integration of newly developed optimization techniques, e.g. new RL algorithms, which are continuously put forward by an active research community. In this paper, we introduce Karolos, a RL framework developed for robotic applications, with a particular focus on transfer scenarios with varying robot-task combinations reflected in a modular environment architecture. In addition, we provide implementations of state-of-the-art RL algorithms along with common learning-facilitating enhancements, as well as an architecture to parallelize environments across multiple processes to significantly speed up experiments. The code is open source and published on GitHub with the aim of promoting research of RL applications in robotics.