Variational Bayesian strategies for high-dimensional, stochastic design problems

TL;DR

This paper introduces a Variational Bayesian framework for efficiently solving high-dimensional, stochastic design optimization problems by recasting them as probabilistic inference tasks, reducing computational costs significantly.

Contribution

It proposes a novel VB-EM scheme that identifies low-dimensional sensitive directions in the design space, improving optimization under uncertainty.

Findings

Validated on problems with around 1,000 variables

Achieved computational costs of about 100 forward model calls

Provided accurate approximations assessed by information-theoretic metrics

Abstract

This paper is concerned with a lesser-studied problem in the context of model-based, uncertainty quantification (UQ), that of optimization/design/control under uncertainty. The solution of such problems is hindered not only by the usual difficulties encountered in UQ tasks (e.g. the high computational cost of each forward simulation, the large number of random variables) but also by the need to solve a nonlinear optimization problem involving large numbers of design variables and potentially constraints. We propose a framework that is suitable for a large class of such problems and is based on the idea of recasting them as probabilistic inference tasks. To that end, we propose a Variational Bayesian (VB) formulation and an iterative VB-Expectation-Maximization scheme that is also capable of identifying a low-dimensional set of directions in the design space, along which, the objective exhibits the largest sensitivity. We demonstrate the validity of the proposed approach in the context of two numerical examples involving $\mathcal{O}(10^3)$ random and design variables. In all cases considered the cost of the computations in terms of calls to the forward model was of the order $\mathcal{O}(10^2)$. The accuracy of the approximations provided is assessed by appropriate information-theoretic metrics.