Activation Landscapes as a Topological Summary of Neural Network Performance

TL;DR

This paper employs topological data analysis to examine how neural network activations evolve across layers, revealing insights into the complexity and transformation of data within deep neural networks.

Contribution

It introduces a novel application of persistent homology and persistence landscapes to analyze neural network activations, providing new visualization and statistical tools.

Findings

Topological complexity often increases with training.

Topological complexity does not necessarily decrease across layers.

Persistence landscapes serve as effective features for analysis.

Abstract

We use topological data analysis (TDA) to study how data transforms as it passes through successive layers of a deep neural network (DNN). We compute the persistent homology of the activation data for each layer of the network and summarize this information using persistence landscapes. The resulting feature map provides both an informative visual- ization of the network and a kernel for statistical analysis and machine learning. We observe that the topological complexity often increases with training and that the topological complexity does not decrease with each layer.