Diffusion-Modeled Reinforcement Learning for Carbon and Risk-Aware Microgrid Optimization

TL;DR

This paper presents DiffCarl, a diffusion-based reinforcement learning algorithm that improves microgrid operation by reducing costs and emissions while managing uncertainty and operational risks.

Contribution

It introduces a novel diffusion-model integrated DRL framework for carbon- and risk-aware microgrid optimization, enhancing policy expressiveness and adaptability.

Findings

Achieves 2.3-30.1% lower operational costs

Reduces carbon emissions by 28.7% compared to non-carbon-aware methods

Outperforms classic and state-of-the-art DRL algorithms in experiments

Abstract

This paper introduces DiffCarl, a diffusion-modeled carbon- and risk-aware reinforcement learning algorithm for intelligent operation of multi-microgrid systems. With the growing integration of renewables and increasing system complexity, microgrid communities face significant challenges in real-time energy scheduling and optimization under uncertainty. DiffCarl integrates a diffusion model into a deep reinforcement learning (DRL) framework to enable adaptive energy scheduling under uncertainty and explicitly account for carbon emissions and operational risk. By learning action distributions through a denoising generation process, DiffCarl enhances DRL policy expressiveness and enables carbon- and risk-aware scheduling in dynamic and uncertain microgrid environments. Extensive experimental studies demonstrate that it outperforms classic algorithms and state-of-the-art DRL solutions, with 2.3-30.1% lower operational cost. It also achieves 28.7% lower carbon emissions than those of its carbon-unaware variant and reduces performance variability. These results highlight DiffCarl as a practical and forward-looking solution. Its flexible design allows efficient adaptation to different system configurations and objectives to support real-world deployment in evolving energy systems.