Centre manifold theorem for maps along manifolds of fixed points

TL;DR

This paper establishes a centre manifold theorem for maps along manifolds of fixed points and applies it to analyze gradient descent in two-layer matrix factorization with large steps.

Contribution

It introduces a centre manifold theorem tailored for maps along fixed point manifolds and applies it to gradient descent analysis in matrix factorization.

Findings

Proves a centre manifold theorem for maps along manifolds of fixed points.

Provides insights into gradient descent dynamics with large step sizes.

Applies the theorem to two-layer matrix factorization problems.

Abstract

We prove a centre manifold theorem for a map along a manifold-with-boundary of fixed points, and provide an application to the study of gradient descent with large step size on two-layer matrix factorisation problems.