Supervisory Output Prediction for Bilinear Systems by Reinforcement Learning

TL;DR

This paper presents a reinforcement learning-based supervisory scheme for real-time selection of optimal output prediction models in bilinear systems, improving model accuracy during operation.

Contribution

It introduces a novel reinforcement learning approach for adaptive model selection and domain partitioning in bilinear systems, with proven convergence properties.

Findings

The scheme converges to the best prediction model and partition.

Simulation results demonstrate improved temperature prediction accuracy.

The method adapts to changing operating conditions effectively.

Abstract

Online output prediction is an indispensable part of any model predictive control implementation, especially when simplifications of the underlying physical model have been considered and/or the operating conditions change quite often. Furthermore, the selection of an output prediction model is strongly related to the data available, while designing/altering the data collection process may not be an option. Thus, in several scenarios, selecting the most appropriate prediction model needs to be performed during runtime. To this end, this paper introduces a supervisory output prediction scheme, tailored specifically for input-output stable bilinear systems, that intends on automating the process of selecting the most appropriate prediction model during runtime. The selection process is based upon a reinforcement-learning scheme, where prediction models are selected according to their prior prediction performance. An additional selection process is concerned with appropriately partitioning the control-inputs' domain in order to also allow for switched-system approximations of the original bilinear dynamics. We show analytically that the proposed scheme converges (in probability) to the best model and partition. We finally demonstrate these properties through simulations of temperature prediction in residential buildings.