Dynamically Stable Poincar\'e Embeddings for Neural Manifolds

TL;DR

This paper introduces Ricci flow-assisted neural networks that leverage the stability of Poincaré embeddings to improve robustness and performance in image classification tasks by dynamically evolving neural manifolds.

Contribution

It proposes a novel integration of Ricci flow with neural networks to create stable hyperbolic embeddings that enhance robustness and accuracy.

Findings

Ricci flow converges exponentially to hyperbolic metrics on neural manifolds.

Dynamically stable Poincaré embeddings improve neural network robustness.

The proposed method outperforms Euclidean models on image classification.

Abstract

In a Riemannian manifold, the Ricci flow is a partial differential equation for evolving the metric to become more regular. We hope that topological structures from such metrics may be used to assist in the tasks of machine learning. However, this part of the work is still missing. In this paper, we propose Ricci flow assisted Eucl2Hyp2Eucl neural networks that bridge this gap between the Ricci flow and deep neural networks by mapping neural manifolds from the Euclidean space to the dynamically stable Poincar\'e ball and then back to the Euclidean space. As a result, we prove that, if initial metrics have an $L^2$-norm perturbation which deviates from the Hyperbolic metric on the Poincar\'e ball, the scaled Ricci-DeTurck flow of such metrics smoothly and exponentially converges to the Hyperbolic metric. Specifically, the role of the Ricci flow is to serve as naturally evolving to the stable Poincar\'e ball. For such dynamically stable neural manifolds under the Ricci flow, the convergence of neural networks embedded with such manifolds is not susceptible to perturbations. And we show that Ricci flow assisted Eucl2Hyp2Eucl neural networks outperform with their all Euclidean counterparts on image classification tasks.