Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generation

TL;DR

This paper introduces a framework and dataset for generating physically consistent structural engineering models using large language models, ensuring simulation readiness and reducing invalid outputs.

Contribution

It presents a novel physics-aware modeling framework, a domain-specific dataset CivilInstruct, and a verification benchmark MBEval to improve the reliability of automated structural model generation.

Findings

Significant reduction in non-conforming outputs with the proposed fine-tuning strategy.

Improved performance on verification metrics over baseline models.

Demonstrated effectiveness of the framework in generating simulation-ready models.

Abstract

Structural modeling is a fundamental component of computational engineering science, in which even minor physical inconsistencies or specification violations may invalidate downstream simulations. The potential of large language models (LLMs) for automatic generation of modeling code has been demonstrated. However, non-executable or physically inconsistent outputs remain prevalent under stringent engineering constraints. A framework for physics-consistent automatic building modeling is therefore proposed, integrating domain knowledge construction, constraint-oriented model alignment, and verification-driven evaluation. CivilInstruct is introduced as a domain-specific dataset that formalizes structural engineering knowledge and constraint reasoning to enable simulation-ready model generation. A two-stage fine-tuning strategy is further employed to enforce constraint satisfaction and application programming interface compliance, substantially reducing hallucinated and non-conforming outputs. MBEval is presented as a verification-driven benchmark that evaluates executability and structural dynamics consistency through closed-loop validation. Experimental results show consistent improvements over baselines across rigorous verification metrics. Our code is available at https://github.com/Jovanqing/AutoBM.