Persistent Homology Captures the Generalization of Neural Networks Without A Validation Set

TL;DR

This paper proposes using Persistent Homology, an algebraic topology tool, to monitor neural network training and estimate generalization error intrinsically, eliminating the need for a validation set.

Contribution

It introduces a novel approach applying Persistent Homology to neural network training, correlating topological changes with generalization performance.

Findings

PH diagram distance correlates with validation accuracy

Intrinsic generalization estimation possible without holdout set

Applicable across different architectures and datasets

Abstract

The training of neural networks is usually monitored with a validation (holdout) set to estimate the generalization of the model. This is done instead of measuring intrinsic properties of the model to determine whether it is learning appropriately. In this work, we suggest studying the training of neural networks with Algebraic Topology, specifically Persistent Homology (PH). Using simplicial complex representations of neural networks, we study the PH diagram distance evolution on the neural network learning process with different architectures and several datasets. Results show that the PH diagram distance between consecutive neural network states correlates with the validation accuracy, implying that the generalization error of a neural network could be intrinsically estimated without any holdout set.