Grammar-Aware Literate Generative Mathematical Programming with Compiler-in-the-Loop

TL;DR

This paper presents SyntAGM, a grammar-aware system that translates natural language into mathematical programming models using compiler feedback and language models, improving accuracy and efficiency.

Contribution

Introducing SyntAGM, a novel system that leverages compiler diagnostics and grammar-awareness for natural language to AML translation, with a generate--compile--assess--revise loop.

Findings

SyntAGM achieves competitive accuracy against baseline methods.

SyntAGM demonstrates superior token efficiency and lower latency.

The system effectively uses compiler feedback for model validation.

Abstract

This work investigates generative mathematical programming through the lens of Algebraic Modelling Languages (AMLs) and compiler-guided model synthesis. By leveraging PyOPL, an OPL-like AML compiler that provides detailed syntax diagnostics, we introduce SyntAGM, an end-to-end system that translates natural language problem descriptions into PyOPL models via a generate--compile--assess--revise loop. SyntAGM is grammar-aware thanks to in-context exposure to the PyOPL BNF grammar, and benefits from few-shot retrieval of literate PyOPL model exemplars. To obtain a valid PyOPL model that matches the problem description, SyntAGM mobilises compiler feedback and an LLM-based alignment judge. In a comparative study against established prompting baselines SyntAGM achieves competitive accuracy with superior token, cost, and latency profiles.