Input Convex Encoder-Only Transformer for Fast and Gradient-Stable MPC in Building Demand Response

TL;DR

This paper introduces the Input-Convex Encoder-only Transformer (IC-EoT), a novel neural network architecture that enables fast, gradient-stable, and convex optimization-based Model Predictive Control for building demand response, significantly reducing solver times.

Contribution

The paper proposes the IC-EoT architecture, combining Transformer parallel processing with input convexity, to improve real-time MPC performance over existing recurrent convex neural networks.

Findings

IC-EoT is immune to gradient instability affecting recurrent ICNNs.

IC-EoT reduces MPC solver times by up to 8.3 times compared to IC-LSTM.

IC-EoT maintains predictive accuracy comparable to existing models.

Abstract

Learning-based Model Predictive Control (MPC) has emerged as a powerful strategy for building demand response. However, its practical deployment is often hindered by the non-convex optimization problems induced by standard neural network models. These problems lead to long solver times and suboptimal solutions, making real-time control over long horizons challenging. While Input Convex Neural Networks (ICNNs), such as Input-Convex Long Short-Term Memorys (IC-LSTMs), are developed to address the convexity issue, their recurrent architectures suffer from high computational cost and gradient instability as the prediction horizon increases. To overcome these limitations, this paper introduces the Input-Convex Encoder-only Transformer (IC-EoT), a novel architecture that synergizes the parallel processing capabilities of the Transformer with the guaranteed tractability of input convexity. The IC-EoT was developed and evaluated in a high-fidelity co-simulation framework using the Energym Python library to interface with the EnergyPlus building simulator, and compared against its recurrent convex counterpart (IC-LSTM) and standard non-convex models. The results demonstrate that the IC-EoT is structurally immune to the gradient instability that affects recurrent ICNNs while maintaining comparable predictive accuracy. More critically, it substantially reduces MPC solver times; this speed advantage grows with the prediction horizon, with the IC-EoT proving 2.7 to 8.3 times faster than the IC-LSTM across horizons spanning from one to eight hours. This leap in computational efficiency makes the IC-EoT a robust and practical solution, enabling effective, real-time MPC for building energy management under realistic horizon decision-making scenarios.