Singular Learning Theory for Factor Analysis

TL;DR

This paper applies algebraic methods to analyze the singular learning properties of factor analysis models, providing bounds and formulas for their learning coefficients which influence Bayesian model selection.

Contribution

It introduces algebraic techniques to compute and bound learning coefficients for factor analysis, a widely used latent variable model, advancing understanding of their asymptotic Bayesian behavior.

Findings

Provided a general upper bound for learning coefficients in factor analysis.

Derived exact formulas for specific factor analysis cases.

Highlighted the impact of singularities on Bayesian model selection.

Abstract

Watanabe's singular learning theory provides a framework for asymptotic analysis of Bayesian model selection for statistical models with singularities, where traditional statistical regularity assumptions fail. Learning coefficients, also known as real log canonical thresholds, play a central role in singular learning, as they govern the asymptotic behavior of Bayesian marginal likelihood integrals in settings where the Laplace approximations used for regular statistical models are not applicable. Learning coefficients are algebraic invariants that quantify the geometric complexity of a model and reveal how the singular structure impacts the model's generalization properties. In this paper, we apply algebraic methods to study the learning coefficients of factor analysis models, which are widely used latent variable models for continuously distributed data. Our main results provide a general upper bound for the learning coefficients as well as exact formulas for specific cases.