Autoformulation of Mathematical Optimization Models Using LLMs

TL;DR

This paper introduces a novel method that uses Large Language Models combined with Monte-Carlo Tree Search to automatically generate solver-ready optimization models from natural language descriptions, addressing key challenges in autoformulation.

Contribution

It presents a new approach leveraging LLMs and Monte-Carlo Tree Search with symbolic pruning and evaluation to improve autoformulation of optimization models from natural language.

Findings

Significant performance improvements over baseline methods.

Effective exploration of the formulation hypothesis space.

Enhanced accuracy in model correctness evaluation.

Abstract

Mathematical optimization is fundamental to decision-making across diverse domains, from operations research to healthcare. Yet, translating real-world problems into optimization models remains a difficult task, often demanding specialized expertise. This paper approaches the problem of $\textit{autoformulation}$: the automated creation of solver-ready optimization models from natural language problem descriptions. We identify three core challenges of autoformulation: $\textit{(1)}$ the vast, problem-dependent hypothesis space, $\textit{(2)}$ efficient and diverse exploration of this space under uncertainty, and $\textit{(3)}$ evaluation of formulation correctness against problem description. To address these challenges, we present a novel method leveraging $\textit{Large Language Models}$ (LLMs) with $\textit{Monte-Carlo Tree Search}$, exploiting the hierarchical nature of optimization modeling to generate and systematically explore possible formulations. To enhance search efficiency, we introduce symbolic pruning to eliminate trivially equivalent search paths (branches), and employ LLM-based evaluation of partial formulations to guide search. Empirical analysis on linear and mixed-integer programming benchmarks demonstrates our method's effectiveness, with significant performance gains from both LLM-based value estimation and symbolic pruning techniques.