Multifractality and percolation in the coupling space of perceptrons

TL;DR

This paper analyzes the complex structure of the perceptron’s coupling space, revealing a multifractal partition influenced by input patterns, and connects these properties to storage and generalization capabilities.

Contribution

It provides an analytical calculation of the multifractal spectrum of perceptron coupling space using replica formalism, linking geometric phase transitions to symmetry breaking.

Findings

Multifractal spectrum $f(\alpha)$ can be computed analytically.

Storage capacity relates to properties of the multifractal spectrum.

Numerical studies support the analytical results for binary couplings.

Abstract

The coupling space of perceptrons with continuous as well as with binary weights gets partitioned into a disordered multifractal by a set of $p=\gamma N$ random input patterns. The multifractal spectrum $f(\alpha)$ can be calculated analytically using the replica formalism. The storage capacity and the generalization behaviour of the perceptron are shown to be related to properties of $f(\alpha)$ which are correctly described within the replica symmetric ansatz. Replica symmetry breaking is interpreted geometrically as a transition from percolating to non-percolating cells. The existence of empty cells gives rise to singularities in the multifractal spectrum. The analytical results for binary couplings are corroborated by numerical studies.