Multifractal Analysis of the Coupling Space of Feed-Forward Neural Networks

TL;DR

This paper investigates the multifractal structure of the coupling space in feed-forward neural networks, revealing phase transitions and instabilities linked to the network's capacity and output behavior.

Contribution

It analytically characterizes the multifractal spectrum of the perceptron’s coupling space and relates phase transitions to percolation phenomena.

Findings

Multifractal spectrum $f(\alpha)$ can be calculated analytically.

Phase transitions correspond to crossover in cell size percolation.

Negative moment instabilities relate to VC-dimension.

Abstract

Random input patterns induce a partition of the coupling space of feed-forward neural networks into different cells according to the generated output sequence. For the perceptron this partition forms a random multifractal for which the spectrum $f(\alpha)$ can be calculated analytically using the replica trick. Phase transition in the multifractal spectrum correspond to the crossover from percolating to non-percolating cell sizes. Instabilities of negative moments are related to the VC-dimension.