Large Language Model-Based Automatic Formulation for Stochastic Optimization Models

TL;DR

This study explores how large language models, especially ChatGPT, can automatically formulate and solve various stochastic optimization problems from natural language descriptions, demonstrating promising results with structured prompting and multi-agent collaboration.

Contribution

It introduces a systematic approach using LLMs for formulating stochastic optimization models from natural language, including novel prompt designs and a soft-scoring metric for evaluation.

Findings

GPT-4-Turbo outperforms GPT-3.5 in partial scoring on most problems.

Structured prompts significantly improve model performance and reduce errors.

Multi-agent collaboration enhances the quality of formulated models.

Abstract

This paper presents an integrated systematic study of the performance of large language models (LLMs), specifically ChatGPT, for automatically formulating and solving Stochastic Optimization (SO) problems from natural language descriptions. Focusing on three key categories, individual chance-constrained models, joint chance-constrained models, and two-stage stochastic mixed-integer linear programming models, we design several prompts that guide ChatGPT through structured tasks using chain-of-thought and agentic reasoning. We introduce a novel soft-scoring metric that evaluates the structural quality and partial correctness of generated models, addressing the limitations of canonical and execution-based accuracy metrics. Across a diverse set of SO problems, GPT-4-Turbo achieves better partial scores than GPT-3.5 variants except for individual chance-constrained problems. Structured prompts significantly outperform simple prompting, reducing extra-element generation and improving objective matching, although extra-element generation remains a nontrivial task. Our findings reveal that with well-engineered prompts and multi-agent collaboration, LLMs can facilitate SO formulations, paving the way for intelligent, language-driven modeling pipelines for SO in practice.