High Dimensional Mode Hunting Using Pettiest Components Analysis

TL;DR

This paper introduces pettiest components analysis, emphasizing the importance of smallest variance components for mode detection, and demonstrates its superiority over principal components in identifying modes and generating better digit representations.

Contribution

The paper presents a novel method called pettiest components analysis, showing its effectiveness in mode detection and data representation, outperforming principal components in specific applications.

Findings

Pettiest components produce minimal volume boxes for normal and Laplace distributions.

Pettiest components outperform principal components in mode detection.

Modes identified with pettiest components generate better digit images in MNIST.

Abstract

Principal components analysis has been used to reduce the dimensionality of datasets for a long time. In this paper, we will demonstrate that in mode detection the components of smallest variance, the pettiest components, are more important. We prove that for a multivariate normal or Laplace distribution, we obtain boxes of optimal volume by implementing "pettiest component analysis", in the sense that their volume is minimal over all possible boxes with the same number of dimensions and fixed probability. This reduction in volume produces an information gain that is measured using active information. We illustrate our results with a simulation and a search for modal patterns of digitized images of hand-written numbers using the famous MNIST database; in both cases pettiest components work better than their competitors. In fact, we show that modes obtained with pettiest components generate better written digits for MNIST than principal components.