Enabling Large Language Models to Perform Power System Simulations with Previously Unseen Tools: A Case of Daline

TL;DR

This paper presents a modular framework that enables large language models to perform power system simulations using previously unseen tools, significantly improving their accuracy and demonstrating potential as research assistants in power system analysis.

Contribution

The work introduces a novel modular framework that integrates power system expertise with LLMs, allowing them to perform simulations with new tools like Daline, which was previously inaccessible to LLMs.

Findings

GPT-4o's simulation accuracy increased from 0% to 96.07%.

Outperformed ChatGPT-4o web interface's 33.8% accuracy.

Validated on 34 Daline simulation tasks.

Abstract

The integration of experiment technologies with large language models (LLMs) is transforming scientific research, offering AI capabilities beyond specialized problem-solving to becoming research assistants for human scientists. In power systems, simulations are essential for research. However, LLMs face significant challenges in power system simulations due to limited pre-existing knowledge and the complexity of power grids. To address this issue, this work proposes a modular framework that integrates expertise from both the power system and LLM domains. This framework enhances LLMs' ability to perform power system simulations on previously unseen tools. Validated using 34 simulation tasks in Daline, a (optimal) power flow simulation and linearization toolbox not yet exposed to LLMs, the proposed framework improved GPT-4o's simulation coding accuracy from 0% to 96.07%, also outperforming the ChatGPT-4o web interface's 33.8% accuracy (with the entire knowledge base uploaded). These results highlight the potential of LLMs as research assistants in power systems.