Rotation to Sparse Loadings using $L^p$ Losses and Related Inference Problems

TL;DR

This paper introduces a novel oblique rotation method for exploratory factor analysis based on $L^p$ loss functions, improving interpretability, accuracy, and efficiency especially for sparse loadings, with theoretical guarantees and practical validation.

Contribution

It proposes a new $L^p$ loss-based rotation method for EFA, along with model selection, inference procedures, and an efficient algorithm, demonstrating superior performance for sparse models.

Findings

Outperforms traditional rotation and regularised estimation in accuracy

Offers computational efficiency for sparse loading matrices

Provides theoretical guarantees for consistency and inference

Abstract

Researchers have widely used exploratory factor analysis (EFA) to learn the latent structure underlying multivariate data. Rotation and regularised estimation are two classes of methods in EFA that they often use to find interpretable loading matrices. In this paper we propose a new family of oblique rotations based on component-wise $L^p$ loss functions $(0 < p\leq 1)$ that is closely related to an $L^p$ regularised estimator. We develop model selection and post-selection inference procedures based on the proposed rotation method. When the true loading matrix is sparse, the proposed method tends to outperform traditional rotation and regularised estimation methods in terms of statistical accuracy and computational cost. Since the proposed loss functions are nonsmooth, we develop an iteratively reweighted gradient projection algorithm for solving the optimisation problem. We also develop theoretical results that establish the statistical consistency of the estimation, model selection, and post-selection inference. We evaluate the proposed method and compare it with regularised estimation and traditional rotation methods via simulation studies. We further illustrate it using an application to the Big Five personality assessment.