The Continuous Stochastic Gradient Method: Part II -- Application and Numerics

TL;DR

This paper analyzes the continuous stochastic gradient (CSG) method, demonstrating its effectiveness in topology optimization problems with many variables and complex integrals, and providing initial convergence rate estimates validated by experiments.

Contribution

It introduces a hybrid stochastic-gradient method that retains past gradient information, enabling solutions to complex large-scale optimization problems previously unsolvable.

Findings

CSG effectively handles large-scale topology optimization.

Convergence rate estimates are supported by numerical experiments.

CSG outperforms traditional methods in complex integral-based objectives.

Abstract

In this contribution, we present a numerical analysis of the continuous stochastic gradient (CSG) method, including applications from topology optimization and convergence rates. In contrast to standard stochastic gradient optimization schemes, CSG does not discard old gradient samples from previous iterations. Instead, design dependent integration weights are calculated to form a linear combination as an approximation to the true gradient at the current design. As the approximation error vanishes in the course of the iterations, CSG represents a hybrid approach, starting off like a purely stochastic method and behaving like a full gradient scheme in the limit. In this work, the efficiency of CSG is demonstrated for practically relevant applications from topology optimization. These settings are characterized by both, a large number of optimization variables \textit{and} an objective function, whose evaluation requires the numerical computation of multiple integrals concatenated in a nonlinear fashion. Such problems could not be solved by any existing optimization method before. Lastly, with regards to convergence rates, first estimates are provided and confirmed with the help of numerical experiments.