RLlib: Abstractions for Distributed Reinforcement Learning

TL;DR

RLlib introduces a scalable library that provides abstractions for distributed reinforcement learning, enabling efficient, high-performance implementation of diverse RL algorithms through composable primitives.

Contribution

The paper presents RLlib, a library offering scalable, reusable primitives for distributed RL, facilitating flexible and efficient algorithm development.

Findings

RLlib achieves high scalability and performance in distributed RL tasks.

The library enables substantial code reuse across different RL algorithms.

RLlib's abstractions simplify the implementation of complex RL algorithms.

Abstract

Reinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation. We argue for distributing RL components in a composable way by adapting algorithms for top-down hierarchical control, thereby encapsulating parallelism and resource requirements within short-running compute tasks. We demonstrate the benefits of this principle through RLlib: a library that provides scalable software primitives for RL. These primitives enable a broad range of algorithms to be implemented with high performance, scalability, and substantial code reuse. RLlib is available at https://rllib.io/.