Projected Stochastic Gradients for Convex Constrained Problems in Hilbert Spaces

TL;DR

This paper proves convergence properties of a projected stochastic gradient method for convex optimization problems in Hilbert spaces, including PDE-constrained cases, with theoretical guarantees and numerical demonstrations.

Contribution

It introduces a convergence analysis for projected stochastic gradients in Hilbert spaces, covering both convex and strongly convex cases, with applications to PDE-constrained optimization.

Findings

Weak convergence to minimizers in convex case

Strong convergence to the unique solution in strongly convex case

Numerical validation of theoretical results

Abstract

Convergence of a projected stochastic gradient algorithm is demonstrated for convex objective functionals with convex constraint sets in Hilbert spaces. In the convex case, the sequence of iterates ${u_n}$ converges weakly to a point in the set of minimizers with probability one. In the strongly convex case, the sequence converges strongly to the unique optimum with probability one. An application to a class of PDE constrained problems with a convex objective, convex constraint and random elliptic PDE constraints is shown. Theoretical results are demonstrated numerically.