Computation of the Maximum Likelihood estimator in low-rank Factor Analysis

TL;DR

This paper introduces a scalable, fast algorithm for maximum likelihood estimation in low-rank factor analysis, reformulating the problem as a nonsmooth semidefinite optimization and demonstrating superior performance over existing methods.

Contribution

It reformulates the low-rank ML factor analysis as a nonsmooth semidefinite problem and develops efficient algorithms with theoretical guarantees and broad applicability.

Findings

Algorithms scale to large problems

Effective even with rank-deficient covariance matrices

Outperforms existing state-of-the-art methods

Abstract

Factor analysis, a classical multivariate statistical technique is popularly used as a fundamental tool for dimensionality reduction in statistics, econometrics and data science. Estimation is often carried out via the Maximum Likelihood (ML) principle, which seeks to maximize the likelihood under the assumption that the positive definite covariance matrix can be decomposed as the sum of a low rank positive semidefinite matrix and a diagonal matrix with nonnegative entries. This leads to a challenging rank constrained nonconvex optimization problem. We reformulate the low rank ML Factor Analysis problem as a nonlinear nonsmooth semidefinite optimization problem, study various structural properties of this reformulation and propose fast and scalable algorithms based on difference of convex (DC) optimization. Our approach has computational guarantees, gracefully scales to large problems, is applicable to situations where the sample covariance matrix is rank deficient and adapts to variants of the ML problem with additional constraints on the problem parameters. Our numerical experiments demonstrate the significant usefulness of our approach over existing state-of-the-art approaches.