Local and global topological complexity measures OF ReLU neural network functions

TL;DR

This paper introduces new topological complexity measures for ReLU neural network functions using a generalized Morse theory, providing a framework for analysis and demonstrating that local complexity can be arbitrarily high.

Contribution

It develops a novel topological framework for analyzing ReLU networks by constructing a canonical polytopal complex and deformation retract, enabling complexity calculations.

Findings

Constructed a canonical polytopal complex for ReLU networks

Defined local and global topological complexity measures

Showed local complexity can be arbitrarily high

Abstract

We apply a generalized piecewise-linear (PL) version of Morse theory due to Grunert-Kuhnel-Rote to define and study new local and global notions of topological complexity for fully-connected feedforward ReLU neural network functions, F: R^n -> R. Along the way, we show how to construct, for each such F, a canonical polytopal complex K(F) and a deformation retract of the domain onto K(F), yielding a convenient compact model for performing calculations. We also give a construction showing that local complexity can be arbitrarily high.