Toward Automated Simulation Research Workflow through LLM Prompt Engineering Design

TL;DR

This paper demonstrates the potential of Large Language Models, especially GPT-4o, to automate entire simulation research workflows through prompt engineering, enabling long-term autonomous research with minimal human intervention.

Contribution

It introduces an autonomous simulation agent (ASA) powered by LLMs that can perform end-to-end simulation research tasks based on human research plans, showcasing near-flawless long-task execution.

Findings

ASA-GPT-4o achieved near-flawless execution on research missions.

The automation process can be iterated for up to 20 cycles without human input.

The study discusses self-validation and task management traits of ASA.

Abstract

The advent of Large Language Models (LLMs) has created new opportunities for the automation of scientific research spanning both experimental processes and computational simulations. This study explores the feasibility of constructing an autonomous simulation agent (ASA) powered by LLMs through prompt engineering and automated program design to automate the entire simulation research process according to a human-provided research plan. This process includes experimental design, remote upload and simulation execution, data analysis, and report compilation. Using a well-studied simulation problem of polymer chain conformations as a test case, we assessed the long-task completion and reliability of ASAs powered by different LLMs, including GPT-4o, Claude-3.5, etc. Our findings revealed that ASA-GPT-4o achieved near-flawless execution on designated research missions, underscoring the potential of methods like ASA to achieve automation in simulation research processes to enhance research efficiency. The outlined automation can be iteratively performed for up to 20 cycles without human intervention, illustrating the potential of ASA for long-task workflow automation. Additionally, we discussed the intrinsic traits of ASA in managing extensive tasks, focusing on self-validation mechanisms, and the balance between local attention and global oversight.