Smart Finite State Devices: A Modeling Framework for Demand Response Technologies

TL;DR

This paper presents a modeling framework using Markov Decision Processes for various types of demand response devices, aiming to optimize their control strategies in future distribution market scenarios.

Contribution

It introduces a novel MDP-based modeling approach for different demand response devices, enabling analysis of optimal control strategies in stochastic market conditions.

Findings

MDP models effectively represent device behaviors

Optimal control strategies vary by device type

Framework supports future demand response market analysis

Abstract

We introduce and analyze Markov Decision Process (MDP) machines to model individual devices which are expected to participate in future demand-response markets on distribution grids. We differentiate devices into the following four types: (a) optional loads that can be shed, e.g. light dimming; (b) deferrable loads that can be delayed, e.g. dishwashers; (c) controllable loads with inertia, e.g. thermostatically-controlled loads, whose task is to maintain an auxiliary characteristic (temperature) within pre-defined margins; and (d) storage devices that can alternate between charging and generating. Our analysis of the devices seeks to find their optimal price-taking control strategy under a given stochastic model of the distribution market.