Least Volume Analysis

TL;DR

This paper presents Least Volume (LV), a novel regularization method for autoencoders that reduces latent dimensions by leveraging geometric intuition, with extensions to non-Euclidean spaces and label integration.

Contribution

It introduces LV and GLV methods, providing theoretical insights, practical algorithms, and demonstrating their effectiveness in dimension reduction and representation learning.

Findings

LV effectively reduces latent dimensions in autoencoders.

GLV enables label integration and improves representation quality.

LV and GLV enhance understanding of data topology and disentanglement.

Abstract

This paper introduces Least Volume (LV)--a simple yet effective regularization method inspired by geometric intuition--that reduces the number of latent dimensions required by an autoencoder without prior knowledge of the dataset's intrinsic dimensionality. We show that its effectiveness depends on the Lipschitz continuity of the decoder, prove that Principal Component Analysis (PCA) is a linear special case, and demonstrate that LV induces a PCA-like importance ordering in nonlinear models. We extend LV to non-Euclidean settings as Generalized Least Volume (GLV), enabling the integration of label information into the latent representation. To support implementation, we also develop an accompanying Dynamic Pruning algorithm. We evaluate LV on several benchmark problems, demonstrating its effectiveness in dimension reduction. Leveraging this, we reveal the role of low-dimensional latent spaces in data sampling and disentangled representation, and use them to probe the varying topological complexity of various datasets. GLV is further applied to labeled datasets, where it induces a contrastive learning effect in representations of discrete labels. On a continuous-label airfoil dataset, it produces representations that lead to smooth changes in aerodynamic performance, thereby stabilizing downstream optimization.