Transferring Reinforcement Learning for DC-DC Buck Converter Control via Duty Ratio Mapping: From Simulation to Implementation

TL;DR

This paper presents a novel transfer methodology using duty ratio mapping to successfully implement a deep reinforcement learning controller from simulation to real-world DC-DC buck converter systems, addressing the sim-to-real gap.

Contribution

It introduces a duty ratio mapping approach for transferring RL control strategies from simulation to real hardware in power electronics, a novel solution to the sim-to-real challenge.

Findings

Successful real-world implementation of DRL controller

Improved performance through the proposed transfer methodology

Experimental validation confirms effectiveness

Abstract

Reinforcement learning (RL) control approach with application into power electronics systems has become an emerging topic whilst the sim-to-real issue remains a challenging problem as very few results can be referred to in the literature. Indeed, due to the inevitable mismatch between simulation models and real-life systems, offline trained RL control strategies may sustain unexpected hurdles in practical implementation during transferring procedure. As the main contribution of this paper, a transferring methodology via a delicately designed duty ratio mapping (DRM) is proposed for a DC-DC buck converter. Then, a detailed sim-to-real process is presented to enable the implementation of a model-free deep reinforcement learning (DRL) controller. The feasibility and effectiveness of the proposed methodology are demonstrated by comparative experimental studies.