A multifractal phase-space analysis of perceptrons with biased patterns

TL;DR

This paper analyzes how input and output biases affect the storage and generalization capabilities of perceptrons using a multifractal phase-space approach, revealing that input bias is negligible unless zero and uncovering a new two-level generalization scenario.

Contribution

It introduces a multifractal analysis of perceptrons with biased patterns and presents a novel algorithm for Gibbs learning to compare with theoretical predictions.

Findings

Input bias is irrelevant unless zero.

Output bias leads to a two-level generalization scenario.

Analytical results align with simulations.

Abstract

We calculate the multifractal spectrum of the partition of the coupling space of a perceptron induced by random input-output pairs with non-zero mean. From the results we infer the influence of the input and output bias respectively on both the storage and generalization properties of the network. It turns out that the value of the input bias is irrelevant as long as it is different from zero. The generalization problem with output bias is new and shows an interesting two-level scenario. To compare our analytical results with simulations we introduce a simple and efficient algorithm to implement Gibbs learning.