Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels

TL;DR

This paper introduces a unified theoretical framework to analyze inference performance in densely connected single-layer networks with correlated patterns, linking perceptron learning and linear vector channels.

Contribution

It develops a novel analytical approach based on singular value bases and eigenvalue spectra, unifying analysis of perceptron learning and linear vector channels.

Findings

Characterizes inference performance using a single function of two variables.

Links to existing perceptron and Gaussian linear channel analysis methods.

Demonstrates application to a nontrivial problem.

Abstract

A framework to analyze inference performance in densely connected single-layer feed-forward networks is developed for situations where a given data set is composed of correlated patterns. The framework is based on the assumption that the left and right singular value bases of the given pattern matrix are generated independently and uniformly from Haar measures. This assumption makes it possible to characterize the objective system by a single function of two variables which is determined by the eigenvalue spectrum of the cross-correlation matrix of the pattern matrix. Links to existing methods for analysis of perceptron learning and Gaussian linear vector channels and an application to a simple but nontrivial problem are also shown.