Hybrid Subspace Learning for High-Dimensional Data

TL;DR

This paper introduces a hybrid subspace learning method for high-dimensional data that combines low-dimensional embedding with original features, improving feature extraction and data reconstruction.

Contribution

It proposes a novel hybrid dimensionality reduction model that allows some features to remain in the original space while others are embedded in a low-dimensional subspace, addressing limitations of traditional methods.

Findings

More accurate latent space estimation

Lower reconstruction error compared to existing methods

Effective feature extraction in gene expression and video datasets

Abstract

The high-dimensional data setting, in which p >> n, is a challenging statistical paradigm that appears in many real-world problems. In this setting, learning a compact, low-dimensional representation of the data can substantially help distinguish signal from noise. One way to achieve this goal is to perform subspace learning to estimate a small set of latent features that capture the majority of the variance in the original data. Most existing subspace learning models, such as PCA, assume that the data can be fully represented by its embedding in one or more latent subspaces. However, in this work, we argue that this assumption is not suitable for many high-dimensional datasets; often only some variables can easily be projected to a low-dimensional space. We propose a hybrid dimensionality reduction technique in which some features are mapped to a low-dimensional subspace while others remain in the original space. Our model leads to more accurate estimation of the latent space and lower reconstruction error. We present a simple optimization procedure for the resulting biconvex problem and show synthetic data results that demonstrate the advantages of our approach over existing methods. Finally, we demonstrate the effectiveness of this method for extracting meaningful features from both gene expression and video background subtraction datasets.