Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters

TL;DR

This paper introduces a novel skip training method for Multi-Agent Reinforcement Learning controllers in Wave Energy Converters, significantly improving energy capture efficiency over traditional controllers.

Contribution

It proposes a new skip training approach and hybrid initialization method for MARL, enhancing convergence and performance in complex wave energy control tasks.

Findings

Double-digit energy efficiency gains over baseline

Skip training improves convergence and performance

Hybrid initialization accelerates training process

Abstract

Recent Wave Energy Converters (WEC) are equipped with multiple legs and generators to maximize energy generation. Traditional controllers have shown limitations to capture complex wave patterns and the controllers must efficiently maximize the energy capture. This paper introduces a Multi-Agent Reinforcement Learning controller (MARL), which outperforms the traditionally used spring damper controller. Our initial studies show that the complex nature of problems makes it hard for training to converge. Hence, we propose a novel skip training approach which enables the MARL training to overcome performance saturation and converge to more optimum controllers compared to default MARL training, boosting power generation. We also present another novel hybrid training initialization (STHTI) approach, where the individual agents of the MARL controllers can be initially trained against the baseline Spring Damper (SD) controller individually and then be trained one agent at a time or all together in future iterations to accelerate convergence. We achieved double-digit gains in energy efficiency over the baseline Spring Damper controller with the proposed MARL controllers using the Asynchronous Advantage Actor-Critic (A3C) algorithm.