Multinomial Multiple Correspondence Analysis

TL;DR

This paper introduces multinomial multiple correspondence analysis (MMCA), a probabilistic extension of MCA that models categorical data with a low-rank structure and employs regularization to prevent overfitting, enhancing interpretability and applicability to high-dimensional data.

Contribution

It develops a probabilistic model for MCA using maximum likelihood, introduces an efficient estimation algorithm, and incorporates regularization to handle high-dimensional data.

Findings

The proposed MMCA effectively captures dependencies in categorical data.

Regularization prevents overfitting and improves model stability.

The method is suitable for high-dimensional categorical datasets.

Abstract

Relations between categorical variables can be analyzed conveniently by multiple correspondence analysis (MCA). %It is well suited to discover relations that may exist between categories of different variables. The graphical representation of MCA results in so-called biplots makes it easy to interpret the most important associations. However, a major drawback of MCA is that it does not have an underlying probability model for an individual selecting a category on a variable. In this paper, we propose such probability model called multinomial multiple correspondence analysis (MMCA) that combines the underlying low-rank representation of MCA with maximum likelihood. An efficient majorization algorithm that uses an elegant bound for the second derivative is derived to estimate the parameters. The proposed model can easily lead to overfitting causing some of the parameters to wander of to infinity. We add the nuclear norm penalty to counter this issue and discuss ways of selecting regularization parameters. The proposed approach is well suited to study and vizualise the dependences for high dimensional data.