Model reduction, machine learning based global optimisation for large-scale steady state nonlinear systems

TL;DR

This paper introduces a novel model reduction and machine learning approach combining PCA and neural networks to efficiently optimize large-scale nonlinear systems modeled by PDEs, with strategies to handle non-convexity.

Contribution

It develops a reduced surrogate model using PCA and ANNs, and proposes two strategies to improve optimization efficiency for non-convex neural network activation functions.

Findings

Effective surrogate models for PDE-based systems

Two strategies significantly reduce optimization time

Demonstrated on two case studies

Abstract

Many engineering processes can be accurately modelled using partial differential equations (PDEs), but high dimensionality and non-convexity of the resulting systems pose limitations on their efficient optimisation. In this work, a model reduction, machine-learning methodology combining principal component analysis (PCA) and artificial neural networks (ANNs) is employed to construct a reduced surrogate model, which can then be utilised by advanced deterministic global optimisation algorithms to compute global optimal solutions with theoretical guarantees. However, such optimisation would still be time-consuming due to the high non-convexity of the activation functions inside the reduced ANN structures. To develop a computationally-efficient optimisation framework, we propose two alternative strategies: The first one is a piecewise-affine reformulation of the nonlinear ANN activation functions, while the second one is based on deep rectifier neural networks with ReLU activation function. The performance of the proposed framework is demonstrated through two illustrative case studies.