An Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control

TL;DR

This paper presents a decentralized multi-agent reinforcement learning framework for EV charging control that enhances privacy, reduces network costs, and mitigates transformer overload risks during peak demand periods.

Contribution

It introduces a novel CTDE-DDPG based MARL framework for EV charging that balances privacy preservation with improved network performance.

Findings

Reduces network costs compared to centralized methods

Decreases Peak-to-Average Ratio of demand

Mitigates transformer overload risk during peak hours

Abstract

The increasing trend in adopting electric vehicles (EVs) will significantly impact the residential electricity demand, which results in an increased risk of transformer overload in the distribution grid. To mitigate such risks, there are urgent needs to develop effective EV charging controllers. Currently, the majority of the EV charge controllers are based on a centralized approach for managing individual EVs or a group of EVs. In this paper, we introduce a decentralized Multi-agent Reinforcement Learning (MARL) charging framework that prioritizes the preservation of privacy for EV owners. We employ the Centralized Training Decentralized Execution-Deep Deterministic Policy Gradient (CTDE-DDPG) scheme, which provides valuable information to users during training while maintaining privacy during execution. Our results demonstrate that the CTDE framework improves the performance of the charging network by reducing the network costs. Moreover, we show that the Peak-to-Average Ratio (PAR) of the total demand is reduced, which, in turn, reduces the risk of transformer overload during the peak hours.