Large Language Model enabled Mathematical Modeling

TL;DR

This paper explores how Large Language Models, especially DeepSeek-R1, can be integrated into mathematical modeling for operations research, focusing on reducing hallucinations and improving formulation accuracy in real-world decision-making scenarios.

Contribution

It systematically evaluates DeepSeek-R1's effectiveness in OR benchmarks and introduces strategies to mitigate hallucinations and enhance model reliability.

Findings

DeepSeek-R1 performs well on OR benchmarks.

Mitigation strategies reduce hallucination rates.

Enhanced formulation accuracy in applied OR scenarios.

Abstract

The integration of Large Language Models (LLMs) with optimization modeling offers a promising avenue for advancing decision-making in operations research (OR). Traditional optimization methods,such as linear programming, mixed integer programming, and simulation depend heavily on domain expertise to translate real-world problems into solvable mathematical models. While solvers like Gurobi and COPT are powerful, expert input remains essential for defining objectives, constraints, and variables. This research investigates the potential of LLMs, specifically the DeepSeek-R1 model, to bridge this formulation gap using natural language understanding and code generation. Although prior models like GPT-4, Claude, and Bard have shown strong performance in NLP and reasoning tasks, their high token costs and tendency toward hallucinations limit real-world applicability in supply chain contexts. In contrast, DeepSeek-R1, a cost-efficient and high-performing model trained with reinforcement learning, presents a viable alternative. Despite its success in benchmarks such as LiveCodeBench and Math-500, its effectiveness in applied OR scenarios remains under explored. This study systematically evaluates DeepSeek-R1 across four key OR benchmarks: NL4OPT, IndustryOR, EasyLP, and ComplexOR. Our methodology includes baseline assessments, the development of a hallucination taxonomy, and the application of mitigation strategies like LLM-as-a-Judge, Few-shot Learning (FSL), Tool Calling, and a Multi-agent Framework. These techniques aim to reduce hallucinations, enhance formulation accuracy, and better align model outputs with user intent.