A Framework for Data-Driven Explainability in Mathematical Optimization

TL;DR

This paper proposes a data-driven explainability framework for mathematical optimization solutions, enabling practitioners to understand and trade off solution optimality with interpretability, demonstrated through theoretical analysis and real-world experiments.

Contribution

It introduces a novel explainability criterion for optimization solutions, characterizes its computational complexity, and provides practical algorithms and experiments.

Findings

Explainability can be integrated as a trade-off with optimality.

The explainable shortest path problem is polynomially solvable.

Enforcing explainability incurs minimal additional cost in practice.

Abstract

Advancements in mathematical programming have made it possible to efficiently tackle large-scale real-world problems that were deemed intractable just a few decades ago. However, provably optimal solutions may not be accepted due to the perception of optimization software as a black box. Although well understood by scientists, this lacks easy accessibility for practitioners. Hence, we advocate for introducing the explainability of a solution as another evaluation criterion, next to its objective value, which enables us to find trade-off solutions between these two criteria. Explainability is attained by comparing against (not necessarily optimal) solutions that were implemented in similar situations in the past. Thus, solutions are preferred that exhibit similar features. Although we prove that already in simple cases the explainable model is NP-hard, we characterize relevant polynomially solvable cases such as the explainable shortest path problem. Our numerical experiments on both artificial as well as real-world road networks show the resulting Pareto front. It turns out that the cost of enforcing explainability can be very small.