PCA of probability measures: Sparse and Dense sampling regimes

TL;DR

This paper investigates PCA on probability measures using Hilbert space embeddings, deriving convergence rates in a double asymptotic regime and identifying a transition from sparse to dense sampling regimes.

Contribution

It introduces a theoretical framework for PCA on multiple measures with sample size analysis and establishes minimax optimal rates in the dense regime.

Findings

Convergence rates of $n^{-1/2} + m^{-eta}$ for covariance and PCA risk.

Identification of a sparse-to-dense transition in sampling regimes.

Validation of theoretical rates through numerical experiments.

Abstract

A common approach to perform PCA on probability measures is to embed them into a Hilbert space where standard functional PCA techniques apply. While convergence rates for estimating the embedding of a single measure from $m$ samples are well understood, the literature has not addressed the setting involving multiple measures. In this paper, we study PCA in a double asymptotic regime where $n$ probability measures are observed, each through $m$ samples. We derive convergence rates of the form $n^{-1/2} + m^{-\alpha}$ for the empirical covariance operator and the PCA excess risk, where $\alpha>0$ depends on the chosen embedding. This characterizes the relationship between the number $n$ of measures and the number $m$ of samples per measure, revealing a sparse (small $m$) to dense (large $m$) transition in the convergence behavior. Moreover, we prove that the dense-regime rate is minimax optimal for the empirical covariance error. Our numerical experiments validate these theoretical rates and demonstrate that appropriate subsampling preserves PCA accuracy while reducing computational cost.