Solving Dynamic Programming with Supremum Terms in the Objective and Application to Optimal Battery Scheduling for Electricity Consumers Subject to Demand Charges

TL;DR

This paper develops a novel dynamic programming approach for problems with supremum terms in the objective, enabling optimal battery scheduling for electricity consumers with demand charges, including stochastic scenarios.

Contribution

It introduces a method to handle supremum terms in dynamic programming by transforming them into forward separable objectives with augmented states, applicable to stochastic problems.

Findings

Proposed a new dynamic programming framework for supremum-based objectives.

Validated the approach on battery scheduling with demand charges.

Demonstrated effectiveness with a stochastic electricity usage model.

Abstract

In this paper, we consider the problem of dynamic programming when supremum terms appear in the objective function. Such terms can represent overhead costs associated with the underlying state variables. Specifically, this form of optimization problem can be used to represent optimal scheduling of batteries such as the Tesla Powerwall for electrical consumers subject to demand charges - a charge based on the maximum rate of electricity consumption. These demand charges reflect the cost to the utility of building and maintaining generating capacity. Unfortunately, we show that dynamic programming problems with supremum terms do not satisfy the principle of optimality. However, we also show that the supremum is a special case of the class of forward separable objective functions. To solve the dynamic programming problem, we propose a general class of optimization problems with forward separable objectives. We then show that for any problem in this class, there exists an augmented-state dynamic programming problem which satisfies the principle of optimality and the solutions to which yield solutions to the original forward separable problem. We further generalize this approach to stochastic dynamic programming problems and apply the results to the problem of optimal battery scheduling with demand charges using a databased stochastic model for electricity usage and solar generation by the consumer.