Sparse Factor Analysis for Categorical Data with the Group-Sparse Generalized Singular Value Decomposition

TL;DR

This paper introduces a novel group-sparse GSVD algorithm that enhances categorical data analysis by producing interpretable, sparse components in correspondence analysis, improving interpretability and variable selection.

Contribution

The paper develops a new group-sparse GSVD method that incorporates orthogonality and sparsity constraints, specifically tailored for categorical data analysis in correspondence analysis.

Findings

The algorithm successfully sparsifies components in multiple correspondence analysis.

Applications demonstrate improved interpretability of categorical data components.

Method adapts to various CA-based analyses with effective sparsification.

Abstract

Correspondence analysis, multiple correspondence analysis and their discriminant counterparts (i.e., discriminant simple correspondence analysis and discriminant multiple correspondence analysis) are methods of choice for analyzing multivariate categorical data. In these methods, variables are integrated into optimal components computed as linear combinations whose weights are obtained from a generalized singular value decomposition (GSVD) that integrates specific metric constraints on the rows and columns of the original data matrix. The weights of the linear combinations are, in turn, used to interpret the components, and this interpretation is facilitated when components are 1) pairwise orthogonal and 2) when the values of the weights are either large or small but not intermediate-a pattern called a simple or a sparse structure. To obtain such simple configurations, the optimization problem solved by the GSVD is extended to include new constraints that implement component orthogonality and sparse weights. Because multiple correspondence analysis represents qualitative variables by a set of binary variables, an additional group constraint is added to the optimization problem in order to sparsify the whole set representing one qualitative variable. This new algorithm-called group-sparse GSVD (gsGSVD)-integrates these constraints via an iterative projection scheme onto the intersection of subspaces where each subspace implements a specific constraint. In this paper, we expose this new algorithm and show how it can be adapted to the sparsification of simple and multiple correspondence analysis, and illustrate its applications with the analysis of four different data sets-each illustrating the sparsification of a particular CA-based analysis.