Distributed Ensembles of Reinforcement Learning Agents for Electricity Control

TL;DR

This paper introduces an ensemble approach of multiple reinforcement learning agents to improve decision-making in electricity control, demonstrating significant performance gains and better reproducibility through extensive experiments.

Contribution

It presents a novel ensemble method for RL agents in electricity control, with comprehensive experimental validation across multiple environments.

Findings

Ensemble of 4 agents improves rewards by 46%.

Reproducibility increases by a factor of 3.6.

Method enables efficient parallel training and prediction.

Abstract

Deep Reinforcement Learning (or just "RL") is gaining popularity for industrial and research applications. However, it still suffers from some key limits slowing down its widespread adoption. Its performance is sensitive to initial conditions and non-determinism. To unlock those challenges, we propose a procedure for building ensembles of RL agents to efficiently build better local decisions toward long-term cumulated rewards. For the first time, hundreds of experiments have been done to compare different ensemble constructions procedures in 2 electricity control environments. We discovered an ensemble of 4 agents improves accumulated rewards by 46%, improves reproducibility by a factor of 3.6, and can naturally and efficiently train and predict in parallel on GPUs and CPUs.