An application of reinforcement learning to residential energy storage under real-time pricing

TL;DR

This paper explores how reinforcement learning can optimize residential energy storage control under real-time electricity pricing to enhance demand response and reduce costs.

Contribution

It introduces a reinforcement learning approach specifically designed for battery management in real-time pricing environments, addressing a gap in existing demand response algorithms.

Findings

RL-based control improves peak shaving efficiency

Reduces electricity bills in simulated environments

Adapts to fluctuating real-time prices effectively

Abstract

With the proliferation of advanced metering infrastructure (AMI), more real-time data is available to electric utilities and consumers. Such high volumes of data facilitate innovative electricity rate structures beyond flat-rate and time-of-use (TOU) tariffs. One such innovation is real-time pricing (RTP), in which the wholesale market-clearing price is passed directly to the consumer on an hour-by-hour basis. While rare, RTP exists in parts of the United States and has been observed to reduce electric bills. Although these reductions are largely incidental, RTP may represent an opportunity for large-scale peak shaving, demand response, and economic efficiency when paired with intelligent control systems. Algorithms controlling flexible loads and energy storage have been deployed for demand response elsewhere in the literature, but few studies have investigated these algorithms in an RTP environment. If properly optimized, the dynamic between RTP and intelligent control has the potential to counteract the unwelcome spikes and dips of demand driven by growing penetration of distributed renewable generation and electric vehicles (EV). This paper presents a simple reinforcement learning (RL) application for optimal battery control subject to an RTP signal.