Topological Data Analysis of Neural Network Layer Representations

TL;DR

This study explores how topological features of data are preserved or transformed within neural network layers using persistent homology, revealing early layer topology preservation and effects of activation functions.

Contribution

It applies topological data analysis to neural network representations, highlighting how topology evolves across layers and the impact of activation functions.

Findings

Early layers approximate homeomorphisms of data

Deeper layers significantly alter data topology

Bijective activation functions help preserve topological features

Abstract

This paper is a cursory study on how topological features are preserved within the internal representations of neural network layers. Using techniques from topological data analysis, namely persistent homology, the topological features of a simple feedforward neural network's layer representations of a modified torus with a Klein bottle-like twist were computed. The network appeared to approximate homeomorphisms in early layers, before significantly changing the topology of the data in deeper layers. The resulting noise hampered the ability of persistent homology to compute these features, however similar topological features seemed to persist longer in a network with a bijective activation function.