A Conservation Law Method in Optimization

TL;DR

This paper introduces algorithms inspired by a conservation law in physics to find local and global minima in nonconvex optimization, utilizing a frictionless Newtonian approach with symplectic Euler schemes and demonstrating high-speed convergence through theoretical analysis.

Contribution

It presents a novel physics-inspired optimization method based on a conservation law and symplectic Euler schemes, with theoretical convergence analysis and experimental validation.

Findings

High-speed convergence in the proposed algorithms

Effective optimization on convex and nonconvex functions

Validation through experiments in high-dimensional settings

Abstract

We propose some algorithms to find local minima in nonconvex optimization and to obtain global minima in some degree from the Newton Second Law without friction. With the key observation of the velocity observable and controllable in the motion, the algorithms simulate the Newton Second Law without friction based on symplectic Euler scheme. From the intuitive analysis of analytical solution, we give a theoretical analysis for the high-speed convergence in the algorithm proposed. Finally, we propose the experiments for strongly convex function, non-strongly convex function and nonconvex function in high-dimension.