Clustering Enabled Few-Shot Load Forecasting

TL;DR

This paper introduces a clustering-based approach combined with a two-phase LSTM model to improve load forecasting for new users with limited data, leveraging historical data from existing users to enhance prediction accuracy.

Contribution

It proposes a novel clustering and two-phase LSTM framework that effectively utilizes historical data for few-shot load forecasting, addressing data scarcity issues.

Findings

Outperforms traditional LSTM in limited data scenarios

Effective clustering improves feature extraction for load profiles

Validated on real-world and synthetic datasets

Abstract

While the advanced machine learning algorithms are effective in load forecasting, they often suffer from low data utilization, and hence their superior performance relies on massive datasets. This motivates us to design machine learning algorithms with improved data utilization. Specifically, we consider the load forecasting for a new user in the system by observing only few shots (data points) of its energy consumption. This task is challenging since the limited samples are insufficient to exploit the temporal characteristics, essential for load forecasting. Nonetheless, we notice that there are not too many temporal characteristics for residential loads due to the limited kinds of human lifestyle. Hence, we propose to utilize the historical load profile data from existing users to conduct effective clustering, which mitigates the challenges brought by the limited samples. Specifically, we first design a feature extraction clustering method for categorizing historical data. Then, inheriting the prior knowledge from the clustering results, we propose a two-phase Long Short Term Memory (LSTM) model to conduct load forecasting for new users. The proposed method outperforms the traditional LSTM model, especially when the training sample size fails to cover a whole period (i.e., 24 hours in our task). Extensive case studies on two real-world datasets and one synthetic dataset verify the effectiveness and efficiency of our method.