Path homologies of deep feedforward networks

TL;DR

This paper characterizes the directed homology structures of fully-connected feedforward neural networks, revealing how their topology relates to architecture and layer configuration, and providing a foundation for comparing different neural network designs.

Contribution

First exact characterization of directed homology in neural networks, linking it to simplicial homology and layer structure, advancing understanding of neural network topology.

Findings

Directed flag homology reduces to simplicial homology of the underlying graph.

Path homology is non-trivial in higher dimensions and depends on layer configuration.

Provides a basis for comparing neural network architectures via homological differences.

Abstract

We provide a characterization of two types of directed homology for fully-connected, feedforward neural network architectures. These exact characterizations of the directed homology structure of a neural network architecture are the first of their kind. We show that the directed flag homology of deep networks reduces to computing the simplicial homology of the underlying undirected graph, which is explicitly given by Euler characteristic computations. We also show that the path homology of these networks is non-trivial in higher dimensions and depends on the number and size of the layers within the network. These results provide a foundation for investigating homological differences between neural network architectures and their realized structure as implied by their parameters.