Distributional Principal Autoencoders

TL;DR

Distributional Principal Autoencoders (DPA) aim to achieve dimension reduction while preserving the original data distribution, ensuring reconstructed data are identically distributed as the original, regardless of the reduced dimension.

Contribution

The paper introduces DPA, a novel autoencoder that matches the conditional data distribution given latent variables, enabling distribution-preserving dimension reduction.

Findings

DPA successfully reconstructs original data distribution across climate, single-cell, and image datasets.

DPA embeddings preserve meaningful data structures like seasonal cycles and cell types.

Numerical results demonstrate DPA's practical feasibility and effectiveness.

Abstract

Dimension reduction techniques usually lose information in the sense that reconstructed data are not identical to the original data. However, we argue that it is possible to have reconstructed data identically distributed as the original data, irrespective of the retained dimension or the specific mapping. This can be achieved by learning a distributional model that matches the conditional distribution of data given its low-dimensional latent variables. Motivated by this, we propose Distributional Principal Autoencoder (DPA) that consists of an encoder that maps high-dimensional data to low-dimensional latent variables and a decoder that maps the latent variables back to the data space. For reducing the dimension, the DPA encoder aims to minimise the unexplained variability of the data with an adaptive choice of the latent dimension. For reconstructing data, the DPA decoder aims to match the conditional distribution of all data that are mapped to a certain latent value, thus ensuring that the reconstructed data retains the original data distribution. Our numerical results on climate data, single-cell data, and image benchmarks demonstrate the practical feasibility and success of the approach in reconstructing the original distribution of the data. DPA embeddings are shown to preserve meaningful structures of data such as the seasonal cycle for precipitations and cell types for gene expression.