Gradient descent globally solves average-case non-resonant physical design problems

TL;DR

This paper demonstrates that gradient descent can globally solve average-case non-resonant physical design problems with bi-affine constraints, under certain stability assumptions, using random matrix theory.

Contribution

It introduces criteria for gradient descent convergence in large-scale physical design problems and applies random matrix theory to identify problem ensembles where this applies.

Findings

Gradient descent converges to approximate global optima in large non-resonant physical problems.

The paper establishes criteria for convergence based on stability assumptions.

Ensembles of problems satisfying these criteria are characterized using random matrix theory.

Abstract

Optimization problems occurring in a wide variety of physical design problems, including but not limited to optical engineering, quantum control, structural engineering, involve minimization of a simple cost function of the state of the system (e.g. the optical fields, the quantum state) while being constrained by the physics of the system. The physics constraints often makes such problems non-convex and thus only locally solvable, leaving open the question of finding the globally optimal design. In this paper, I consider design problems whose physics is described by bi-affine equality constraints, and show that under assumptions on the stability of these constraints and the physical system being non-resonant, gradient descent globally solves a typical physical design problem. The key technical contributions of this paper are (i) outline a criteria that ensure the convergence of gradient descent to an approximate global optima in the limit of large problem sizes, and (ii) use random matrix theory to outline ensembles of physically motivated problems which, on an average, satisfy this convergence criteria.