Applying Fine-Tuned LLMs for Reducing Data Needs in Load Profile Analysis

TL;DR

This paper introduces a two-stage fine-tuning approach for large language models to effectively restore missing load profile data in power systems, reducing data needs and improving efficiency.

Contribution

It proposes a novel fine-tuning strategy for LLMs like GPT-3.5 to accurately restore missing data in load profiles with limited data, outperforming traditional models.

Findings

Fine-tuned LLMs achieve comparable accuracy to specialized models.

Prompt engineering enhances restoration performance.

Few-shot learning reduces data and computational requirements.

Abstract

This paper presents a novel method for utilizing fine-tuned Large Language Models (LLMs) to minimize data requirements in load profile analysis, demonstrated through the restoration of missing data in power system load profiles. A two-stage fine-tuning strategy is proposed to adapt a pre-trained LLMs, i.e., GPT-3.5, for missing data restoration tasks. Through empirical evaluation, we demonstrate the effectiveness of the fine-tuned model in accurately restoring missing data, achieving comparable performance to state-of-the-art specifically designed models such as BERT-PIN. Key findings include the importance of prompt engineering and the optimal utilization of fine-tuning samples, highlighting the efficiency of few-shot learning in transferring knowledge from general user cases to specific target users. Furthermore, the proposed approach demonstrates notable cost-effectiveness and time efficiency compared to training models from scratch, making it a practical solution for scenarios with limited data availability and computing resources. This research has significant potential for application to other power system load profile analysis tasks. Consequently, it advances the use of LLMs in power system analytics, offering promising implications for enhancing the resilience and efficiency of power distribution systems.