Towards Affordable Energy: A Gymnasium Environment for Electric Utility Demand-Response Programs

TL;DR

This paper introduces DR-Gym, an open-source environment for training demand-response strategies at the utility level, incorporating real-world data, a regime-switching price model, and multi-objective rewards.

Contribution

The paper presents DR-Gym, a novel simulation environment tailored for utility-level demand-response optimization, addressing limitations of existing device-level simulators.

Findings

DR-Gym can simulate realistic demand-response scenarios.

The environment incorporates a calibrated wholesale price model.

Baseline strategies demonstrate the environment's learnability and realism.

Abstract

Extreme weather and volatile wholesale electricity markets expose residential consumers to catastrophic financial risks, yet demand response at the distribution level remains an underutilized tool for grid flexibility and energy affordability. While a demand-response program can shield consumers by issuing financial credits during high-price periods, optimizing this sequential decision-making process presents a unique challenge for reinforcement learning despite the plentiful offline historical smart meter and wholesale pricing data available publicly. Offline historical data fails to capture the dynamic, interactive feedback loop between an electric utility's pricing signals and customer acceptance and adaptation to a demand-response program. To address this, we introduce DR-Gym, an open-source, online Gymnasium-compatible environment designed to train and evaluate demand-response from the electric utility's perspective. Unlike existing device-level energy simulators, our environment focuses on the market-level electric utility setting and provides a rich observational space relevant to the electric utility. The simulator additionally features a regime-switching wholesale price model calibrated to real-world extreme events, alongside physics-based building demand profiles. For our learning signal, we use a configurable, multi-objective reward function for specifying diverse learning objectives. We demonstrate through baseline strategies and data snapshots the capability of our simulator to create realistic and learnable environments.