Addressing Terminal Constraints in Data-Driven Demand Response Scheduling

TL;DR

This paper proposes a novel RL-based scheduling method that integrates Goal-Space Planning with DDPG to efficiently handle terminal constraints in demand response for chemical processes.

Contribution

It introduces a GSP-augmented DDPG approach that improves sample efficiency and constraint satisfaction in long-horizon demand response scheduling.

Findings

The method outperforms standard DDPG in sample efficiency.

It effectively satisfies terminal storage constraints.

Demonstrated on a simulated air separation benchmark.

Abstract

Electrified chemical processes are incentivized by exposure to time-varying electricity markets to operate flexibly, but participating in demand response schemes can require satisfying terminal constraints over long horizons. Specifically, terminal constraints may be required when computing optimal schedules in order to preserve dynamic stability. Model-based optimization methods are computationally costly, and data-driven scheduling via reinforcement learning (RL) faces severe credit-assignment challenges. We integrate Goal-Space Planning (GSP) with Deep Deterministic Policy Gradient (DDPG), using learned temporally abstract models over discrete subgoals to propagate value across extended horizons. Using a simulated air separation benchmark, we demonstrate the proposed approach improves sample efficiency over standard DDPG while satisfying terminal storage constraints, mitigating myopic control behavior.