Energy-Stable Swarm-Based Inertial Algorithms for Optimization

TL;DR

This paper introduces energy-stable swarm algorithms inspired by nonequilibrium thermodynamics, improving global optimization by dynamically coupling agents' inertia and energy dissipation.

Contribution

It develops a novel class of energy-stable algorithms based on a dissipative dynamical system framework for swarm optimization.

Findings

Algorithms preserve or enhance energy dissipation at the discrete level.

Converges toward low local minima, increasing global minimum likelihood.

Outperforms traditional swarm gradient descent methods with fewer agents.

Abstract

We formulate the swarming optimization problem as a weakly coupled, dissipative dynamical system governed by a controlled energy dissipation rate and initial velocities that adhere to the nonequilibrium Onsager principle. In this framework, agents' inertia, positions, and masses are dynamically coupled. To numerically solve the system, we develop a class of efficient, energy-stable algorithms that either preserve or enhance energy dissipation at the discrete level. At equilibrium, the system tends to converge toward one of the lowest local minima explored by the agents, thereby improving the likelihood of identifying the global minimum. Numerical experiments confirm the effectiveness of the proposed approach, demonstrating significant advantages over traditional swarm-based gradient descent methods, especially when operating with a limited number of agents.