On the Topology of Neural Network Superlevel Sets

TL;DR

This paper demonstrates that neural networks with certain activation functions produce outputs with topologically bounded superlevel sets, allowing for architecture-only bounds on their topological complexity.

Contribution

It introduces a class of neural networks with Riccati-type activations that enable architecture-dependent bounds on the topology of superlevel sets and related loci.

Findings

Superlevel sets have topology bounds depending only on network architecture.

Neural network outputs are Pfaffian functions on analytic domains.

Betti numbers of superlevel sets are uniformly bounded by architecture.

Abstract

We show that neural networks with activations satisfying a Riccati-type ordinary differential equation condition, an assumption arising in recent universal approximation results in the uniform topology, produce Pfaffian outputs on analytic domains with format controlled only by the architecture. Consequently, superlevel sets, as well as Lie bracket rank drop loci for neural network parameterized vector fields, admit architecture-only bounds on topological complexity, in particular on total Betti numbers, uniformly over all weights.