Federated Reinforcement Learning for Real-Time Electric Vehicle Charging and Discharging Control

TL;DR

This paper introduces a federated reinforcement learning approach to optimize real-time electric vehicle charging and discharging in dynamic environments, enhancing user benefits while preserving privacy.

Contribution

It develops a novel horizontal federated reinforcement learning method for EV control, addressing behavioral diversity and environmental stochasticity without sharing user profiles.

Findings

Effective in reducing charging costs under real-time pricing

Adapts to diverse user behaviors and dynamic conditions

Maintains privacy by avoiding data sharing

Abstract

With the recent advances in mobile energy storage technologies, electric vehicles (EVs) have become a crucial part of smart grids. When EVs participate in the demand response program, the charging cost can be significantly reduced by taking full advantage of the real-time pricing signals. However, many stochastic factors exist in the dynamic environment, bringing significant challenges to design an optimal charging/discharging control strategy. This paper develops an optimal EV charging/discharging control strategy for different EV users under dynamic environments to maximize EV users' benefits. We first formulate this problem as a Markov decision process (MDP). Then we consider EV users with different behaviors as agents in different environments. Furthermore, a horizontal federated reinforcement learning (HFRL)-based method is proposed to fit various users' behaviors and dynamic environments. This approach can learn an optimal charging/discharging control strategy without sharing users' profiles. Simulation results illustrate that the proposed real-time EV charging/discharging control strategy can perform well among various stochastic factors.