Deep reinforcement learning-based joint real-time energy scheduling for green buildings with heterogeneous battery energy storage devices

TL;DR

This paper introduces a deep reinforcement learning approach for real-time energy scheduling in green buildings, effectively integrating stationary storage and electric vehicles to reduce operational costs.

Contribution

It presents a novel model-free DRL method with degradation modeling and prioritized experience replay for joint scheduling of heterogeneous energy storage devices.

Findings

Achieved up to 40% reduction in operating costs.

Developed accurate degradation and cost models for ESS and EVs.

Enhanced scheduling efficiency with DRL techniques like double networks and dueling mechanisms.

Abstract

Green buildings (GBs) with renewable energy and building energy management systems (BEMS) enable efficient energy use and support sustainable development. Electric vehicles (EVs), as flexible storage resources, enhance system flexibility when integrated with stationary energy storage systems (ESS) for real-time scheduling. However, differing degradation and operational characteristics of ESS and EVs complicate scheduling strategies. This paper proposes a model-free deep reinforcement learning (DRL) method for joint real-time scheduling based on a combined battery system (CBS) integrating ESS and EVs. We develop accurate degradation models and cost estimates, prioritize EV travel demands, and enable collaborative ESS-EV operation under varying conditions. A prediction model optimizes energy interaction between CBS and BEMS. To address heterogeneous states, action coupling, and learning efficiency, the DRL algorithm incorporates double networks, a dueling mechanism, and prioritized experience replay. Experiments show a 37.94 percent to 40.01 percent reduction in operating costs compared to a mixed-integer linear programming (MILP) approach.