Inverse Optimization Without Inverse Optimization: Direct Solution Prediction with Transformer Models

TL;DR

This paper introduces a transformer-based neural network approach for directly predicting solutions to complex combinatorial optimization problems with unknown components, bypassing traditional inverse optimization methods.

Contribution

It proposes a novel end-to-end framework that learns from past solutions and incorporates known constraints, achieving fast and near-optimal solutions for various problems.

Findings

Transformer models outperform LSTM-based methods in complex scenarios.

The approach produces near-optimal solutions in milliseconds.

Effective for problems with unknown objectives and implicit constraints.

Abstract

We present an end-to-end framework for generating solutions to combinatorial optimization problems with unknown components using transformer-based sequence-to-sequence neural networks. Our framework learns directly from past solutions and incorporates the known components, such as hard constraints, via a constraint reasoning module, yielding a constrained learning scheme. The trained model generates new solutions that are structurally similar to past solutions and are guaranteed to respect the known constraints. We apply our approach to three combinatorial optimization problems with unknown components: the knapsack problem with an unknown reward function, the bipartite matching problem with an unknown objective function, and the single-machine scheduling problem with release times and unknown precedence constraints, with the objective of minimizing average completion time. We demonstrate that transformer models have remarkably strong performance and often produce near-optimal solutions in a fraction of a second. They can be particularly effective in the presence of more complex underlying objective functions and unknown implicit constraints compared to an LSTM-based alternative and inverse optimization.