ReLU Neural Networks, Polyhedral Decompositions, and Persistent Homolog

TL;DR

This paper demonstrates that ReLU neural networks induce a polyhedral decomposition of input space that, when combined with persistent homology, can detect topological features of manifolds, revealing a surprising robustness useful for topological data analysis.

Contribution

The paper introduces a novel connection between ReLU networks' geometric structure and topological data analysis, showing robustness of the dual graph for detecting manifold features.

Findings

Dual graph of ReLU networks captures topological signals

Persistent homology applied to network-induced decompositions detects manifold features

This property is consistent across various trained networks

Abstract

A ReLU neural network leads to a finite polyhedral decomposition of input space and a corresponding finite dual graph. We show that while this dual graph is a coarse quantization of input space, it is sufficiently robust that it can be combined with persistent homology to detect homological signals of manifolds in the input space from samples. This property holds for a variety of networks trained for a wide range of purposes that have nothing to do with this topological application. We found this feature to be surprising and interesting; we hope it will also be useful.