Towards Universal Large-Scale Foundational Model for Natural Gas Demand Forecasting

TL;DR

This paper introduces a universal foundation model for natural gas demand forecasting that leverages contrastive learning and industry-specific fine-tuning, significantly improving prediction accuracy over traditional methods.

Contribution

It presents the first foundation model tailored for natural gas demand forecasting, integrating noise filtering and fine-tuning to enhance generalization and accuracy across diverse sectors.

Findings

Outperforms state-of-the-art methods with 3.68% lower MSE

Achieves 6.15% better MASE scores

Demonstrates robustness across large-scale industrial data

Abstract

In the context of global energy strategy, accurate natural gas demand forecasting is crucial for ensuring efficient resource allocation and operational planning. Traditional forecasting methods struggle to cope with the growing complexity and variability of gas consumption patterns across diverse industries and commercial sectors. To address these challenges, we propose the first foundation model specifically tailored for natural gas demand forecasting. Foundation models, known for their ability to generalize across tasks and datasets, offer a robust solution to the limitations of traditional methods, such as the need for separate models for different customer segments and their limited generalization capabilities. Our approach leverages contrastive learning to improve prediction accuracy in real-world scenarios, particularly by tackling issues such as noise in historical consumption data and the potential misclassification of similar data samples, which can lead to degradation in the quaility of the representation and thus the accuracy of downstream forecasting tasks. By integrating advanced noise filtering techniques within the contrastive learning framework, our model enhances the quality of learned representations, leading to more accurate predictions. Furthermore, the model undergoes industry-specific fine-tuning during pretraining, enabling it to better capture the unique characteristics of gas consumption across various sectors. We conducted extensive experiments using a large-scale dataset from ENN Group, which includes data from over 10,000 industrial, commercial, and welfare-related customers across multiple regions. Our model outperformed existing state-of-the-art methods, demonstrating a relative improvement in MSE by 3.68\% and in MASE by 6.15\% compared to the best available model.