Deep Relaxation of Controlled Stochastic Gradient Descent via Singular Perturbations

TL;DR

This paper rigorously analyzes the convergence of a singularly perturbed stochastic differential equation system used to approximate entropic gradient descent in deep learning, revealing the limiting control problem and trajectory behavior.

Contribution

It extends the theoretical understanding of the convergence of stochastic control systems related to deep neural network optimization algorithms.

Findings

Limit of value functions corresponds to an effective control problem.

Trajectories of the perturbed system converge to the effective control system.

Provides rigorous convergence results for the algorithms used by Chaudhari et al.

Abstract

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the Entropic Gradient Descent in the optimization of deep neural networks, via homogenisation. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton-Jacobi-Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls, and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modelled as dynamic controls.