Perceptron capacity revisited: classification ability for correlated patterns

TL;DR

This paper investigates the classification capacity of a single-layer perceptron when patterns are correlated, using advanced analytical methods and proposing a message-passing algorithm for practical implementation.

Contribution

It introduces two novel analytical schemes based on the replica and TAP methods to analyze perceptron capacity with correlated patterns, validated through examples.

Findings

Analytical schemes accurately predict perceptron capacity with correlated patterns.

Validation against known results confirms the methods' validity.

A message-passing algorithm enables practical application of the TAP approach.

Abstract

In this paper, we address the problem of how many randomly labeled patterns can be correctly classified by a single-layer perceptron when the patterns are correlated with each other. In order to solve this problem, two analytical schemes are developed based on the replica method and Thouless-Anderson-Palmer (TAP) approach by utilizing an integral formula concerning random rectangular matrices. The validity and relevance of the developed methodologies are shown for one known result and two example problems. A message-passing algorithm to perform the TAP scheme is also presented.