Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices

TL;DR

This paper develops a deep reinforcement learning approach, using the SAC algorithm, to optimize community battery scheduling under uncertainties in load, PV generation, and energy prices, improving system cost efficiency.

Contribution

It introduces a SAC-based RL framework for community battery management that outperforms other RL algorithms and benchmarks in handling uncertainties.

Findings

SAC achieves superior scheduling performance.

RL effectively manages uncertainties in PV, load, and prices.

Noisy network enhances RL training convergence.

Abstract

In response to the growing uptake of distributed energy resources (DERs), community batteries have emerged as a promising solution to support renewable energy integration, reduce peak load, and enhance grid reliability. This paper presents a deep reinforcement learning (RL) strategy, centered around the soft actor-critic (SAC) algorithm, to schedule a community battery system in the presence of uncertainties, such as solar photovoltaic (PV) generation, local demand, and real-time energy prices. We position the community battery to play a versatile role, in integrating local PV energy, reducing peak load, and exploiting energy price fluctuations for arbitrage, thereby minimizing the system cost. To improve exploration and convergence during RL training, we utilize the noisy network technique. This paper conducts a comparative study of different RL algorithms, including proximal policy optimization (PPO) and deep deterministic policy gradient (DDPG) algorithms, to evaluate their effectiveness in the community battery scheduling problem. The results demonstrate the potential of RL in addressing community battery scheduling challenges and show that the SAC algorithm achieves the best performance compared to RL and optimization benchmarks.