Comparing Inverse Optimization and Machine Learning Methods for Imputing a Convex Objective Function

TL;DR

This paper compares inverse optimization and machine learning methods for imputing convex objective functions, providing insights into when each approach is preferable based on data and problem characteristics.

Contribution

It evaluates the predictive performance of standard ML algorithms against a classical IO model for convex problems, offering practical guidance for method selection.

Findings

ML methods perform competitively with IO in certain scenarios

Training set size influences the choice between ML and IO

External parameter dependence affects method effectiveness

Abstract

Inverse optimization (IO) aims to determine optimization model parameters from observed decisions. However, IO is not part of a data scientist's toolkit in practice, especially as many general-purpose machine learning packages are widely available as an alternative. When encountering IO, practitioners face the question of when, or even whether, investing in developing IO methods is worthwhile. Our paper provides a starting point toward answering these questions, focusing on the problem of imputing the objective function of a parametric convex optimization problem. We compare the predictive performance of three standard supervised machine learning (ML) algorithms (random forest, support vector regression and Gaussian process regression) to the performance of the IO model of Keshavarz, Wang, and Boyd (2011). While the IO literature focuses on the development of methods tailored to particular problem classes, our goal is to evaluate general "out-of-the-box" approaches. Our experiments demonstrate that determining whether to use an ML or IO approach requires considering (i) the training set size, (ii) the dependence of the optimization problem on external parameters, (iii) the level of confidence with regards to the correctness of the optimization prior, and (iv) the number of critical regions in the solution space.