RoME: Domain-Robust Mixture-of-Experts for MILP Solution Prediction across Domains

TL;DR

RoME is a novel mixture-of-experts framework that improves the generalization of MILP solution prediction across multiple domains using robust training strategies, significantly enhancing performance on diverse and real-world problems.

Contribution

Introduces RoME, a domain-robust mixture-of-experts model with a novel training strategy for cross-domain MILP solution prediction, improving generalization and performance.

Findings

Single RoME model trained on three domains improves by 67.7% on five domains.

Cross-domain training enhances generalization to unseen domains.

Pretrained RoME shows measurable gains on real-world MIPLIB instances.

Abstract

Mixed-Integer Linear Programming (MILP) is a fundamental and powerful framework for modeling complex optimization problems across diverse domains. Recently, learning-based methods have shown great promise in accelerating MILP solvers by predicting high-quality solutions. However, most existing approaches are developed and evaluated in single-domain settings, limiting their ability to generalize to unseen problem distributions. This limitation poses a major obstacle to building scalable and general-purpose learning-based solvers. To address this challenge, we introduce RoME, a domain-Robust Mixture-of-Experts framework for predicting MILP solutions across domains. RoME dynamically routes problem instances to specialized experts based on learned task embeddings. The model is trained using a two-level distributionally robust optimization strategy: inter-domain to mitigate global shifts across domains, and intra-domain to enhance local robustness by introducing perturbations on task embeddings. We reveal that cross-domain training not only enhances the model's generalization capability to unseen domains but also improves performance within each individual domain by encouraging the model to capture more general intrinsic combinatorial patterns. Specifically, a single RoME model trained on three domains achieves an average improvement of 67.7% then evaluated on five diverse domains. We further test the pretrained model on MIPLIB in a zero-shot setting, demonstrating its ability to deliver measurable performance gains on challenging real-world instances where existing learning-based approaches often struggle to generalize.