Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization

TL;DR

This paper introduces PROF, a differentiable projection method that enforces operational constraints in neural policies for energy systems, improving efficiency and safety in building and inverter control applications.

Contribution

The paper presents a novel differentiable projection layer for neural policies, ensuring feasibility of actions in energy system control tasks.

Findings

Maintains thermal comfort while improving energy efficiency by 4%.

Perfectly satisfies voltage constraints in inverter control.

Enforces convex operational constraints within neural networks.

Abstract

While reinforcement learning (RL) is gaining popularity in energy systems control, its real-world applications are limited due to the fact that the actions from learned policies may not satisfy functional requirements or be feasible for the underlying physical system. In this work, we propose PROjected Feasibility (PROF), a method to enforce convex operational constraints within neural policies. Specifically, we incorporate a differentiable projection layer within a neural network-based policy to enforce that all learned actions are feasible. We then update the policy end-to-end by propagating gradients through this differentiable projection layer, making the policy cognizant of the operational constraints. We demonstrate our method on two applications: energy-efficient building operation and inverter control. In the building operation setting, we show that PROF maintains thermal comfort requirements while improving energy efficiency by 4% over state-of-the-art methods. In the inverter control setting, PROF perfectly satisfies voltage constraints on the IEEE 37-bus feeder system, as it learns to curtail as little renewable energy as possible within its safety set.