The Voice of Optimization

TL;DR

This paper proposes using interpretable machine learning models, specifically OCTs and OCT-Hs, to understand the decision strategies behind optimal solutions in continuous and mixed-integer convex optimization problems, transforming optimization into a classification task.

Contribution

It introduces a novel approach that redefines optimization as a classification problem using OCTs and OCT-Hs, providing insights into the logic of optimal solutions.

Findings

High accuracy (90%-100%) in predicting optimal strategies.

Predictions lead to low suboptimality or infeasibility even when incorrect.

OCT-Hs perform comparably to neural networks.

Abstract

We introduce the idea that using optimal classification trees (OCTs) and optimal classification trees with-hyperplanes (OCT-Hs), interpretable machine learning algorithms developed by Bertsimas and Dunn [2017, 2018], we are able to obtain insight on the strategy behind the optimal solution in continuous and mixed-integer convex optimization problem as a function of key parameters that affect the problem. In this way, optimization is not a black box anymore. Instead, we redefine optimization as a multiclass classification problem where the predictor gives insights on the logic behind the optimal solution. In other words, OCTs and OCT-Hs give optimization a voice. We show on several realistic examples that the accuracy behind our method is in the 90%-100% range, while even when the predictions are not correct, the degree of suboptimality or infeasibility is very low. We compare optimal strategy predictions of OCTs and OCT-Hs and feedforward neural networks (NNs) and conclude that the performance of OCT-Hs and NNs is comparable. OCTs are somewhat weaker but often competitive. Therefore, our approach provides a novel insightful understanding of optimal strategies to solve a broad class of continuous and mixed-integer optimization problems.

Figures