Principal Orthogonal Latent Components Analysis (POLCA Net)

TL;DR

POLCA Net is a novel neural network approach that extends PCA and LDA capabilities to non-linear data, enabling effective dimensionality reduction, feature orthogonality, and improved classification visualization.

Contribution

It introduces POLCA Net, a new autoencoder-based model that combines specialized loss functions to mimic PCA and LDA in non-linear domains, enhancing feature extraction and visualization.

Findings

Achieves effective non-linear dimensionality reduction.

Provides high-fidelity reconstructions.

Produces latent representations suitable for linear classifiers.

Abstract

Representation learning is a pivotal area in the field of machine learning, focusing on the development of methods to automatically discover the representations or features needed for a given task from raw data. Unlike traditional feature engineering, which requires manual crafting of features, representation learning aims to learn features that are more useful and relevant for tasks such as classification, prediction, and clustering. We introduce Principal Orthogonal Latent Components Analysis Network (POLCA Net), an approach to mimic and extend PCA and LDA capabilities to non-linear domains. POLCA Net combines an autoencoder framework with a set of specialized loss functions to achieve effective dimensionality reduction, orthogonality, variance-based feature sorting, high-fidelity reconstructions, and additionally, when used with classification labels, a latent representation well suited for linear classifiers and low dimensional visualization of class distribution as well.