Asymptotic analysis of the Gaussian kernel matrix for partially noisy data in high dimensions

TL;DR

This paper analyzes the asymptotic behavior of Gaussian kernel matrices in high-dimensional noisy data, proposing a new consistent estimator for low-rank cases by combining Karoui's analysis with constrained low-rank approximation techniques.

Contribution

It introduces a novel estimator that achieves strong consistency in low-rank regimes under partial noise, extending Karoui's asymptotic analysis to practical noisy data scenarios.

Findings

Karoui's asymptotic structure of eigenvectors is confirmed.

Naive estimators are inconsistent under partial noise.

The proposed constrained low-rank approximation yields a consistent estimator.

Abstract

The Gaussian kernel is one of the most important kernels, applicable to many research fields, including scientific computing and data science. In this paper, we present asymptotic analysis of the Gaussian kernel matrix in high dimension under a statistical model of noisy data. The main result is a nice combination of Karoui's asymptotic analysis with procedures of constrained low rank matrix approximations. More specifically, Karouli clarified an important asymptotic structure of the Gaussian kernel matrix, leading to strong consistency of the eigenvectors, though the eigenvalues are inconsistent. This paper focuses on the above results and presents a consistent estimator with the use of the smallest eigenvalue, whenever the target kernel matrix tends to low rank in the asymptotic regime. Importantly, asymptotic analysis is given under a statistical model representing partial noise. Although a naive estimator is inconsistent, applying an optimization method for low rank approximations with constraints, we overcome the difficulty caused by the inconsistency, resulting in a new estimator with strong consistency in rank deficient cases.