Towards Foundation Models for Mixed Integer Linear Programming

TL;DR

This paper proposes a foundation model approach for MILP by training on a diverse, synthetically generated dataset, enabling better generalization across problem classes and improving performance on unseen benchmarks.

Contribution

It introduces MILP-Evolve, a novel LLM-based evolutionary framework for generating diverse MILP problems, and demonstrates the effectiveness of a foundation model approach for MILP generalization.

Findings

Models trained on MILP-Evolve data outperform on unseen problems

Significant improvements on MIPLIB benchmarks

Demonstrates potential of foundation models for MILP applications

Abstract

Mixed Integer Linear Programming (MILP) is essential for modeling complex decision-making problems but faces challenges in computational tractability and requires expert formulation. Current deep learning approaches for MILP focus on specific problem classes and do not generalize to unseen classes. To address this shortcoming, we take a foundation model training approach, where we train a single deep learning model on a diverse set of MILP problems to generalize across problem classes. As existing datasets for MILP lack diversity and volume, we introduce MILP-Evolve, a novel LLM-based evolutionary framework that is capable of generating a large set of diverse MILP classes with an unlimited amount of instances. We study our methodology on three key learning tasks that capture diverse aspects of MILP: (1) integrality gap prediction, (2) learning to branch, and (3) a new task of aligning MILP instances with natural language descriptions. Our empirical results show that models trained on the data generated by MILP-Evolve achieve significant improvements on unseen problems, including MIPLIB benchmarks. Our work highlights the potential of moving towards a foundation model approach for MILP that can generalize to a broad range of MILP applications. Our code and data are publicly available at https://github.com/microsoft/OptiGuide.