Informative Data Projections: A Framework and Two Examples

TL;DR

This paper introduces an information-theoretic approach to designing projection indices for high-dimensional data visualization, enabling user-specific, subjective projections and providing new insights into PCA and robust variants.

Contribution

It proposes a novel, user-dependent projection index based on information theory, with practical algorithms for maximization and applications to PCA and robust PCA variants.

Findings

New projection index effectively distinguishes data structures.

Algorithms successfully maximize the index despite non-convexity.

Empirical results show advantages over traditional PCA and Projection Pursuit methods.

Abstract

Methods for Projection Pursuit aim to facilitate the visual exploration of high-dimensional data by identifying interesting low-dimensional projections. A major challenge is the design of a suitable quality metric of projections, commonly referred to as the projection index, to be maximized by the Projection Pursuit algorithm. In this paper, we introduce a new information-theoretic strategy for tackling this problem, based on quantifying the amount of information the projection conveys to a user given their prior beliefs about the data. The resulting projection index is a subjective quantity, explicitly dependent on the intended user. As a useful illustration, we developed this idea for two particular kinds of prior beliefs. The first kind leads to PCA (Principal Component Analysis), shining new light on when PCA is (not) appropriate. The second kind leads to a novel projection index, the maximization of which can be regarded as a robust variant of PCA. We show how this projection index, though non-convex, can be effectively maximized using a modified power method as well as using a semidefinite programming relaxation. The usefulness of this new projection index is demonstrated in comparative empirical experiments against PCA and a popular Projection Pursuit method.