Critical properties of the SAT/UNSAT transitions in the classification problem of structured data

TL;DR

This paper analyzes the critical SAT/UNSAT phase transitions in structured data classification, comparing supervised and unsupervised approaches within replica theory, revealing distinct phase behaviors.

Contribution

It characterizes the phase transition properties of classification problems under different learning strategies using replica theory, highlighting differences in stability and symmetry-breaking.

Findings

Supervised classification exhibits a SAT/UNSAT transition in a stable replica-symmetric phase.

Unsupervised classification's satisfiability line lies in a full replica-symmetry-broken phase.

Similar phase behavior is observed in learning with a margin.

Abstract

The classification problem of structured data can be solved with different strategies: a supervised learning approach, starting from a labeled training set, and an unsupervised learning one, where only the structure of the patterns in the dataset is used to find a classification compatible with it. The two strategies can be interpreted as extreme cases of a semi-supervised approach to learn multi-view data, relevant for applications. In this paper I study the critical properties of the two storage problems associated with these tasks, in the case of the linear binary classification of doublets of points sharing the same label, within replica theory. While the first approach presents a SAT/UNSAT transition in a (marginally) stable replica-symmetric phase, in the second one the satisfiability line lies in a full replica-symmetry-broken phase. A similar behavior in the problem of learning with a margin is also pointed out.