The Symmetric Perceptron: a Teacher-Student Scenario

TL;DR

This paper studies a symmetric binary Perceptron model in a teacher-student setting, analyzing phase transitions and stability in high-dimensional inference with noise, revealing insights into learning dynamics and metastability.

Contribution

It introduces a novel teacher-student formulation of the symmetric Perceptron and maps its phase diagram, highlighting the role of potential choice and metastability in learning.

Findings

Identification of a second-order instability leading to suboptimal states

Discovery of a first-order transition to full alignment

Analysis of metastability and melting phenomena affecting optimization

Abstract

We introduce and solve a teacher-student formulation of the symmetric binary Perceptron, turning a traditionally storage-oriented model into a planted inference problem with a guaranteed solution at any sample density. We adapt the formulation of the symmetric Perceptron which traditionally considers either the u-shaped potential or the rectangular one, by including labels in both regions. With this formulation, we analyze both the Bayes-optimal regime at for noise-less examples and the effect of thermal noise under two different potential/classification rules. Using annealed and quenched free-entropy calculations in the high-dimensional limit, we map the phase diagram in the three control parameters, namely the sample density $\alpha$, the distance between the origin and one of the symmetric hyperplanes $\kappa$ and temperature $T$, and identify a robust scenario where learning is organized by a second-order instability that creates teacher-correlated suboptimal states, followed by a first-order transition to full alignment. We show how this structure depends on the choice of potential, the interplay between metastability of the suboptimal solution and its melting towards the planted configuration, which is relevant for Monte Carlo-based optimization algorithms.