SOC-Boundary and Battery Aging Aware Hierarchical Coordination of Multiple EV Aggregates Among Multi-stakeholders with Multi-Agent Constrained Deep Reinforcement Learning

TL;DR

This paper presents a hierarchical multi-stakeholder V2G coordination strategy using constrained deep reinforcement learning to optimize renewable energy integration, reduce load fluctuations, and consider battery health in EV management.

Contribution

It introduces a novel multi-agent constrained deep reinforcement learning framework combined with the Proof-of-Stake algorithm for multi-stakeholder EV grid coordination.

Findings

Enhances renewable energy integration and grid stability.

Reduces EV charging costs and battery degradation.

Improves load fluctuation mitigation.

Abstract

As electric vehicles (EV) become more prevalent and advances in electric vehicle electronics continue, vehicle-to-grid (V2G) techniques and large-scale scheduling strategies are increasingly important to promote renewable energy utilization and enhance the stability of the power grid. This study proposes a hierarchical multistakeholder V2G coordination strategy based on safe multi-agent constrained deep reinforcement learning (MCDRL) and the Proof-of-Stake algorithm to optimize benefits for all stakeholders, including the distribution system operator (DSO), electric vehicle aggregators (EVAs) and EV users. For DSO, the strategy addresses load fluctuations and the integration of renewable energy. For EVAs, energy constraints and charging costs are considered. The three critical parameters of battery conditioning, state of charge (SOC), state of power (SOP), and state of health (SOH), are crucial to the participation of EVs in V2G. Hierarchical multi-stakeholder V2G coordination significantly enhances the integration of renewable energy, mitigates load fluctuations, meets the energy demands of the EVAs, and reduces charging costs and battery degradation simultaneously.