PCA-Guided Quantile Sampling: Preserving Data Structure in Large-Scale Subsampling

TL;DR

PCA-Guided Quantile Sampling (PCA QS) is a new method that preserves data structure in large datasets by combining PCA guidance with quantile-based stratified sampling, ensuring representativeness and interpretability.

Contribution

The paper introduces PCA QS, a novel sampling framework that retains original data features while guiding sampling with principal components, supported by theoretical guarantees and empirical validation.

Findings

PCA QS outperforms random sampling in structure preservation.

Theoretical convergence rates for quantile and distributional metrics.

Improved downstream model performance using PCA QS samples.

Abstract

We introduce Principal Component Analysis guided Quantile Sampling (PCA QS), a novel sampling framework designed to preserve both the statistical and geometric structure of large scale datasets. Unlike conventional PCA, which reduces dimensionality at the cost of interpretability, PCA QS retains the original feature space while using leading principal components solely to guide a quantile based stratification scheme. This principled design ensures that sampling remains representative without distorting the underlying data semantics. We establish rigorous theoretical guarantees, deriving convergence rates for empirical quantiles, Kullback Leibler divergence, and Wasserstein distance, thus quantifying the distributional fidelity of PCA QS samples. Practical guidelines for selecting the number of principal components, quantile bins, and sampling rates are provided based on these results. Extensive empirical studies on both synthetic and real-world datasets show that PCA QS consistently outperforms simple random sampling, yielding better structure preservation and improved downstream model performance. Together, these contributions position PCA QS as a scalable, interpretable, and theoretically grounded solution for efficient data summarization in modern machine learning workflows.