Domain-specific ReAct for physics-integrated iterative modeling: A case study of LLM agents for gas path analysis of gas turbines

TL;DR

This paper investigates the use of large language models with tool integration for gas turbine gas path analysis, highlighting the potential and challenges of applying LLMs in complex engineering tasks.

Contribution

It introduces a dual-agent tool-calling framework for LLMs in energy engineering and evaluates multiple models' effectiveness in gas turbine analysis.

Findings

Larger LLMs outperform smaller models in tool usage and parameter extraction.

All models struggle with complex, multi-component problems.

Nearly 100 billion parameter LLMs can meet professional requirements with fine-tuning.

Abstract

This study explores the application of large language models (LLMs) with callable tools in energy and power engineering domain, focusing on gas path analysis of gas turbines. We developed a dual-agent tool-calling process to integrate expert knowledge, predefined tools, and LLM reasoning. We evaluated various LLMs, including LLama3, Qwen1.5 and GPT. Smaller models struggled with tool usage and parameter extraction, while larger models demonstrated favorable capabilities. All models faced challenges with complex, multi-component problems. Based on the test results, we infer that LLMs with nearly 100 billion parameters could meet professional scenario requirements with fine-tuning and advanced prompt design. Continued development are likely to enhance their accuracy and effectiveness, paving the way for more robust AI-driven solutions.