Global Deterministic Optimization with Artificial Neural Networks Embedded

TL;DR

This paper introduces an efficient deterministic global optimization method for neural network embedded problems, leveraging McCormick relaxations and a specialized solver, demonstrating superior performance over traditional solvers in various applications.

Contribution

The paper presents a novel approach combining McCormick relaxations with a dedicated solver for global optimization of neural network models, improving computational efficiency.

Findings

Favorable computational times compared to BARON

Effective in diverse applications including chemical and process optimization

Demonstrates robustness and efficiency of the proposed method

Abstract

Artificial neural networks (ANNs) are used in various applications for data-driven black-box modeling and subsequent optimization. Herein, we present an efficient method for deterministic global optimization of ANN embedded optimization problems. The proposed method is based on relaxations of algorithms using McCormick relaxations in a reduced-space [\textit{SIOPT}, 20 (2009), pp. 573-601] including the convex and concave envelopes of the nonlinear activation function of ANNs. The optimization problem is solved using our in-house global deterministic solver MAiNGO. The performance of the proposed method is shown in four optimization examples: an illustrative function, a fermentation process, a compressor plant and a chemical process optimization. The results show that computational solution time is favorable compared to the global general-purpose optimization solver BARON.