Joint Scheduling of Deferrable and Nondeferrable Demand with Colocated Stochastic Supply

TL;DR

This paper develops an optimal scheduling framework for deferrable and nondeferrable electric demands with colocated stochastic supply, introducing a procrastination principle and a reinforcement learning algorithm for unknown distributions.

Contribution

It introduces the Principle of Procrastination for demand scheduling and proposes a Procrastination Threshold Reinforcement Learning algorithm for unknown stochastic environments.

Findings

Optimal scheduling follows the Principle of Procrastination.

The RL algorithm closely approximates the optimal policy.

The proposed method outperforms standard benchmarks in real data tests.

Abstract

We investigate the problem of serving deferrable and nondeferrable electric demands with colocated stochastic supply and grid-imported electricity. Deferrable demands arrive randomly and can be delayed within their service deadlines. Nondeferrable demands are always present and must be served immediately, but the quantity served depends on the cost of electricity. Colocated supply is stochastic with zero marginal cost. It can be used to meet demand or exported to the grid to maximize profit. The stochasticity of demands and local supply makes optimal scheduling a Markov decision process with continuous (uncountable) state and action spaces. Under deterministic, time-varying, and piecewise-linear retail pricing of electricity, we show that the optimal demand scheduling follows the {\em Principle of Procrastination}, which reduces the infinite-dimensional policy space to a finite-dimensional Euclidean space defined by three procrastination parameters for each deferrable demand. For settings in which the underlying probability distributions are unknown, we propose a {\em Procrastination Threshold Reinforcement Learning} algorithm. Numerical experiments based on real-world test data confirm that the proposed threshold learning algorithm closely approximates the optimal policy and outperforms standard benchmarks.