Soft-margin classification of object manifolds

TL;DR

This paper develops a mean-field theory for soft-margin classifiers applied to neural response manifolds, revealing how regularization influences classification robustness and capacity, including a phase transition phenomenon.

Contribution

It introduces a theoretical framework for understanding soft-margin classification of neural manifolds, extending max-margin analysis to include regularization effects and noise robustness.

Findings

Predicts the probability of classification errors based on regularization.

Describes a phase transition in classification capacity.

Provides a soft-margin analogue of the max-margin classifier capacity.

Abstract

A neural population responding to multiple appearances of a single object defines a manifold in the neural response space. The ability to classify such manifolds is of interest, as object recognition and other computational tasks require a response that is insensitive to variability within a manifold. Linear classification of object manifolds was previously studied for max-margin classifiers. Soft-margin classifiers are a larger class of algorithms and provide an additional regularization parameter used in applications to optimize performance outside the training set by balancing between making fewer training errors and learning more robust classifiers. Here we develop a mean-field theory describing the behavior of soft-margin classifiers applied to object manifolds. Analyzing manifolds with increasing complexity, from points through spheres to general manifolds, a mean-field theory describes the expected value of the linear classifier's norm, as well as the distribution of fields and slack variables. By analyzing the robustness of the learned classification to noise, we can predict the probability of classification errors and their dependence on regularization, demonstrating a finite optimal choice. The theory describes a previously unknown phase transition, corresponding to the disappearance of a non-trivial solution, thus providing a soft version of the well-known classification capacity of max-margin classifiers.