Execution-Verified Reinforcement Learning for Optimization Modeling

TL;DR

EVOM introduces an execution-verified reinforcement learning framework for optimization modeling that leverages solver outcomes as scalar rewards, enabling scalable, solver-agnostic decision intelligence.

Contribution

The paper presents EVOM, a novel framework that uses execution outcomes as rewards, removing supervision needs and enabling cross-solver generalization in optimization modeling.

Findings

EVOM matches or outperforms supervised fine-tuning methods.

Supports zero-shot transfer across different solvers.

Achieves effective low-cost adaptation by continued training with target solvers.

Abstract

Automating optimization modeling with LLMs is a promising path toward scalable decision intelligence, but existing approaches either rely on agentic pipelines built on closed-source LLMs with high inference latency, or fine-tune smaller LLMs using costly process supervision that often overfits to a single solver API. Inspired by reinforcement learning with verifiable rewards, we propose Execution-Verified Optimization Modeling (EVOM), an execution-verified learning framework that treats a mathematical programming solver as a deterministic, interactive verifier. Given a natural-language problem and a target solver, EVOM generates solver-specific code, executes it in a sandboxed harness, and converts execution outcomes into scalar rewards, optimized with GRPO and DAPO in a closed-loop generate-execute-feedback-update process. This outcome-only formulation removes the need for process-level supervision, and enables cross-solver generalization by switching the verification environment rather than reconstructing solver-specific datasets. Experiments on NL4OPT, MAMO, IndustryOR, and OptiBench across Gurobi, OR-Tools, and COPT show that EVOM matches or outperforms process-supervised SFT, supports zero-shot solver transfer, and achieves effective low-cost solver adaptation by continuing training under the target solver backend.