Constraint Learning to Define Trust Regions in Predictive-Model Embedded Optimization

TL;DR

This paper explores the integration of trust region constraints into predictive-model embedded optimization, proposing new methods and demonstrating that trust regions learned via isolation forests improve solution quality and efficiency.

Contribution

It introduces three novel approaches for constructing trust regions in predictive-model embedded optimization, with one method outperforming existing techniques.

Findings

Trust-region constraints are crucial for reliable solutions.

Isolation forest-based trust regions outperform previous methods.

Proposed methods improve both solution quality and computational efficiency.

Abstract

There is a recent proliferation of research on the integration of machine learning and optimization. One expansive area within this research stream is predictive-model embedded optimization, which proposes the use of pre-trained predictive models as surrogates for uncertain or highly complex objective functions. In this setting, features of the predictive models become decision variables in the optimization problem. Despite a recent surge in publications in this area, only a few papers note the importance of incorporating trust region considerations in this decision-making pipeline, i.e., enforcing solutions to be similar to the data used to train the predictive models. Without such constraints, the evaluation of the predictive model at solutions obtained from optimization cannot be trusted and the practicality of the solutions may be unreasonable. In this paper, we provide an overview of the approaches appearing in the literature to construct a trust region, and propose three alternative approaches. Our numerical evaluation highlights that trust-region constraints learned through isolation forests, one of the newly proposed approaches, outperform all previously suggested approaches, both in terms of solution quality and computational time.