Gradient Descent Converges to Minimizers

TL;DR

This paper proves that gradient descent almost surely converges to a local minimizer when initialized randomly, using the Stable Manifold Theorem from dynamical systems theory.

Contribution

It provides a rigorous proof of convergence to local minimizers for gradient descent with random initialization, leveraging dynamical systems theory.

Findings

Gradient descent converges to local minimizers almost surely.

The proof uses the Stable Manifold Theorem.

Convergence is guaranteed under random initialization.

Abstract

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.