Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks

TL;DR

This paper introduces ISMO, an active learning algorithm leveraging deep neural networks for PDE-constrained optimization, demonstrating exponential convergence and superior performance over standard methods through numerical experiments.

Contribution

The paper proposes a novel iterative active learning algorithm, ISMO, that improves PDE-constrained optimization using neural networks with proven exponential convergence.

Findings

ISMO achieves exponential convergence with increasing training samples.

Numerical examples show ISMO outperforms standard neural network surrogate methods.

The feedback loop enhances training data selection for better optimization results.

Abstract

We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems. This algorithm is based on deep neural networks and its key feature is the iterative selection of training data through a feedback loop between deep neural networks and any underlying standard optimization algorithm. Under suitable hypotheses, we show that the resulting optimizers converge exponentially fast (and with exponentially decaying variance), with respect to increasing number of training samples. Numerical examples for optimal control, parameter identification and shape optimization problems for PDEs are provided to validate the proposed theory and to illustrate that ISMO significantly outperforms a standard deep neural network based surrogate optimization algorithm.