Deep Learning as Ricci Flow

TL;DR

This paper proposes a novel geometric perspective on deep neural networks, modeling their transformations as Ricci flows to better understand how they disentangle complex data geometries, with empirical evidence linking Ricci flow behavior to model accuracy.

Contribution

It introduces a computational framework linking DNN transformations to Ricci flow, providing a new geometric approach to analyze and explain deep learning models.

Findings

Ricci flow-like behavior correlates with DNN accuracy

The framework applies to both synthetic and real-world data

Global Ricci network flow can assess a model's ability to disentangle data

Abstract

Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general setting of non-smooth activation functions, such as the rectified linear unit (ReLU), which tend to perform better, is required. Here we propose that the geometric transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow - a tool from differential geometry that evolves a manifold by smoothing its curvature, in order to identify its topology. To illustrate this idea, we present a computational framework to quantify the geometric changes that occur as data passes through successive layers of a DNN, and use this framework to motivate a notion of `global Ricci network flow' that can be used to assess a DNN's ability to disentangle complex data geometries to solve classification problems. By training more than $1,500$ DNN classifiers of different widths and depths on synthetic and real-world data, we show that the strength of global Ricci network flow-like behaviour correlates with accuracy for well-trained DNNs, independently of depth, width and data set. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.