# Discretized Gradient Flow for Manifold Learning in the Space of Embeddings

## TL;DR

This paper introduces a method for gradient descent in the space of smooth embeddings of a manifold into Euclidean space, providing explicit step size bounds based on Riemannian geometry, especially for diffeomorphism-invariant penalties.

## Contribution

It develops an explicit lower bound for gradient step sizes in the infinite-dimensional embedding space, considering the geometric structure of the embedded manifold.

## Key findings

- Derived a step size lower bound using Riemannian geometry.
- Proved the normality of the gradient in the case of diffeomorphism invariance.
- Applied the method to manifold learning scenarios.

## Abstract

Gradient descent, or negative gradient flow, is a standard technique in optimization to find minima of functions. Many implementations of gradient descent rely on discretized versions, i.e., moving in the gradient direction for a set step size, recomputing the gradient, and continuing. In this paper, we present an approach to manifold learning where gradient descent takes place in the infinite dimensional space $\mathcal{E} = {\rm Emb}(M,\mathbb{R}^N)$ of smooth embeddings $\phi$ of a manifold $M$ into $\mathbb{R}^N$. Implementing a discretized version of gradient descent for $P:\mathcal{E}\to {\mathbb R}$, a penalty function that scores an embedding $\phi \in \mathcal{E}$, requires estimating how far we can move in a fixed direction -- the direction of one gradient step -- before leaving the space of smooth embeddings. Our main result is to give an explicit lower bound for this step length in terms of the Riemannian geometry of $\phi(M)$. In particular, we consider the case when the gradient of $P$ is pointwise normal to the embedded manifold $\phi(M)$. We prove this case arises when $P$ is invariant under diffeomorphisms of $M$, a natural condition in manifold learning.