d3rlpy: An Offline Deep Reinforcement Learning Library

TL;DR

d3rlpy is an open-source Python library that facilitates offline and online deep reinforcement learning, supporting multiple algorithms, with a focus on reproducibility through extensive benchmarking on D4RL and Atari datasets.

Contribution

The paper introduces d3rlpy, a comprehensive, well-documented library for offline and online deep RL, emphasizing reproducibility with large-scale benchmarks and detailed experimental results.

Findings

Successful implementation of multiple offline RL algorithms

Extensive benchmarking on D4RL and Atari datasets

Open-source code with detailed experimental scripts

Abstract

In this paper, we introduce d3rlpy, an open-sourced offline deep reinforcement learning (RL) library for Python. d3rlpy supports a set of offline deep RL algorithms as well as off-policy online algorithms via a fully documented plug-and-play API. To address a reproducibility issue, we conduct a large-scale benchmark with D4RL and Atari 2600 dataset to ensure implementation quality and provide experimental scripts and full tables of results. The d3rlpy source code can be found on GitHub: \url{https://github.com/takuseno/d3rlpy}.