Data-Driven Online Interactive Bidding Strategy for Demand Response

TL;DR

This paper proposes a data-driven, online interactive bidding strategy for demand response using deep reinforcement learning, enabling demand aggregators to optimize bids in uncertain electricity markets.

Contribution

It introduces a two-agent deep deterministic policy gradient method for real-time bidding decision-making in demand response, incorporating online learning and market uncertainties.

Findings

The model achieves robust profit maximization across diverse market scenarios.

Online learning improves bidding performance over time.

The approach effectively adapts to market dynamics using historical and recent data.

Abstract

Demand response (DR), as one of the important energy resources in the future's grid, provides the services of peak shaving, enhancing the efficiency of renewable energy utilization with a short response period, and low cost. Various categories of DR are established, e.g. automated DR, incentive DR, emergency DR, and demand bidding. However, with the practical issue of the unawareness of residential and commercial consumers' utility models, the researches about demand bidding aggregator involved in the electricity market are just at the beginning stage. For this issue, the bidding price and bidding quantity are two required decision variables while considering the uncertainties due to the market and participants. In this paper, we determine the bidding and purchasing strategy simultaneously employing the smart meter data and functions. A two-agent deep deterministic policy gradient method is developed to optimize the decisions through learning historical bidding experiences. The online learning further utilizes the daily newest bidding experience attained to ensure trend tracing and self-adaptation. Two environment simulators are adopted for testifying the robustness of the model. The results prove that when facing diverse situations the proposed model can earn the optimal profit via off/online learning the bidding rules and robustly making the proper bid.