Accelerated high-index saddle dynamics method for searching high-index saddle points

TL;DR

This paper introduces an accelerated high-index saddle dynamics (A-HiSD) method that improves convergence speed for computing saddle points, especially in ill-conditioned problems, by integrating the heavy ball method.

Contribution

The paper develops an A-HiSD method with proven stability and faster convergence, extending the original HiSD approach for better performance on challenging problems.

Findings

A-HiSD converges faster than HiSD on ill-conditioned problems.

Theoretical analysis confirms linear stability of A-HiSD.

Numerical experiments validate the acceleration on neural network loss landscapes.

Abstract

The high-index saddle dynamics (HiSD) method [J. Yin, L. Zhang, and P. Zhang, {\it SIAM J. Sci. Comput., }41 (2019), pp.A3576-A3595] serves as an efficient tool for computing index-$k$ saddle points and constructing solution landscapes. Nevertheless, the conventional HiSD method often encounters slow convergence rates on ill-conditioned problems. To address this challenge, we propose an accelerated high-index saddle dynamics (A-HiSD) by incorporating the heavy ball method. We prove the linear stability theory of the continuous A-HiSD, and subsequently estimate the local convergence rate for the discrete A-HiSD. Our analysis demonstrates that the A-HiSD method exhibits a faster convergence rate compared to the conventional HiSD method, especially when dealing with ill-conditioned problems. We also perform various numerical experiments including the loss function of neural network to substantiate the effectiveness and acceleration of the A-HiSD method.