A physics-inspired nonlinear momentum method for gradient descent with applications to inverse photonic design

TL;DR

This paper introduces a physics-inspired nonlinear momentum method for gradient descent, utilizing a Hamiltonian framework with nonlinear kinetic energy and damping to improve convergence in inverse photonic design.

Contribution

It develops a novel Hamiltonian-based nonlinear momentum optimization algorithm inspired by physical systems, enhancing convergence speed and interpretability.

Findings

Faster convergence than classical momentum methods

Effective in inverse photonic design tasks

Provides a physically interpretable optimization framework

Abstract

In this work, a nonlinear momentum method is introduced to enhance the convergence performance of momentum-based gradient optimization algorithms. Classical momentum methods, such as the Heavy Ball method, can be viewed as a dynamical system with quadratic kinetic energy and linear damping. By extending this analogy to non-Newtonian dynamical systems, we construct a Hamiltonian framework for optimization problems. In this framework, nonlinear kinetic energy and nonlinear damping effects naturally emerge. It provides a more flexible and physically interpretable mechanism for optimization algorithms. Specifically, we employ an anharmonic kinetic energy function to capture the inertial effects of accumulated gradient information during the optimization process, while the nonlinear damping mechanism effectively regulates the contribution of momentum during convergence. Numerical experiments show that the proposed method achieves faster convergence compared to classical momentum algorithms, making it particularly suitable for inverse design tasks. Moreover, the Hamiltonian based algorithmic framework may offer physical insights for the development of efficient physics-inspired optimization algorithms.