Metric Flows with Neural Networks

TL;DR

This paper develops a theoretical framework for neural network-driven metric flows in Riemannian geometry, explaining their effectiveness in learning complex geometric structures like Calabi-Yau metrics compared to fixed kernel methods.

Contribution

It introduces a general theory of neural network metric flows, highlighting the importance of feature learning and evolving neural tangent kernels in geometric approximation tasks.

Findings

Fixed kernel regimes lead to poor metric learning.

Finite-width networks with evolving kernels better learn Calabi-Yau metrics.

Neural networks outperform fixed kernel methods in geometric learning tasks.

Abstract

We develop a general theory of flows in the space of Riemannian metrics induced by neural network gradient descent. This is motivated in part by recent advances in approximating Calabi-Yau metrics with neural networks and is enabled by recent advances in understanding flows in the space of neural networks. We derive the corresponding metric flow equations, which are governed by a metric neural tangent kernel, a complicated, non-local object that evolves in time. However, many architectures admit an infinite-width limit in which the kernel becomes fixed and the dynamics simplify. Additional assumptions can induce locality in the flow, which allows for the realization of Perelman's formulation of Ricci flow that was used to resolve the 3d Poincar\'e conjecture. We demonstrate that such fixed kernel regimes lead to poor learning of numerical Calabi-Yau metrics, as is expected since the associated neural networks do not learn features. Conversely, we demonstrate that well-learned numerical metrics at finite-width exhibit an evolving metric-NTK, associated with feature learning. Our theory of neural network metric flows therefore explains why neural networks are better at learning Calabi-Yau metrics than fixed kernel methods, such as the Ricci flow.