Transfer Learning in Deep Learning Models for Building Load Forecasting: Case of Limited Data

TL;DR

This paper introduces a transfer learning framework using Transformer models to improve building load forecasting accuracy in scenarios with limited data, outperforming traditional deep learning models.

Contribution

It proposes a novel Building-to-Building Transfer Learning approach with Transformer models for scarce data scenarios, enhancing load forecasting performance.

Findings

Transfer learning improved accuracy by 56.8%.

Transformer outperformed LSTM and RNN models.

Root mean square error reduced to 0.009 with Transformer.

Abstract

Precise load forecasting in buildings could increase the bill savings potential and facilitate optimized strategies for power generation planning. With the rapid evolution of computer science, data-driven techniques, in particular the Deep Learning models, have become a promising solution for the load forecasting problem. These models have showed accurate forecasting results; however, they need abundance amount of historical data to maintain the performance. Considering the new buildings and buildings with low resolution measuring equipment, it is difficult to get enough historical data from them, leading to poor forecasting performance. In order to adapt Deep Learning models for buildings with limited and scarce data, this paper proposes a Building-to-Building Transfer Learning framework to overcome the problem and enhance the performance of Deep Learning models. The transfer learning approach was applied to a new technique known as Transformer model due to its efficacy in capturing data trends. The performance of the algorithm was tested on a large commercial building with limited data. The result showed that the proposed approach improved the forecasting accuracy by 56.8% compared to the case of conventional deep learning where training from scratch is used. The paper also compared the proposed Transformer model to other sequential deep learning models such as Long-short Term Memory (LSTM) and Recurrent Neural Network (RNN). The accuracy of the transformer model outperformed other models by reducing the root mean square error to 0.009, compared to LSTM with 0.011 and RNN with 0.051.