Generalizing Correspondence Analysis for Applications in Machine Learning

TL;DR

This paper introduces a scalable, neural network-based approach to perform correspondence analysis (CA) for high-dimensional data, enabling visualization and interpretation of data dependencies in large datasets.

Contribution

It provides a novel information-theoretic interpretation of CA via principal inertia components and develops algorithms to estimate them using deep neural networks for large-scale applications.

Findings

Neural network algorithms reliably approximate principal inertia components.

CA embeddings help visualize classification boundaries and training dynamics.

The approach scales to high-dimensional, large datasets.

Abstract

Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies by finding maximally correlated embeddings of pairs of random variables. CA has found applications in fields ranging from epidemiology to social sciences; however, current methods do not scale to large, high-dimensional datasets. In this paper, we provide a novel interpretation of CA in terms of an information-theoretic quantity called the principal inertia components. We show that estimating the principal inertia components, which consists in solving a functional optimization problem over the space of finite variance functions of two random variable, is equivalent to performing CA. We then leverage this insight to design novel algorithms to perform CA at an unprecedented scale. Particularly, we demonstrate how the principal inertia components can be reliably approximated from data using deep neural networks. Finally, we show how these maximally correlated embeddings of pairs of random variables in CA further play a central role in several learning problems including visualization of classification boundary and training process, and underlying recent multi-view and multi-modal learning methods.