Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation

TL;DR

The paper introduces LHSD, a spectral filtering method for robust local intrinsic dimension estimation that effectively handles high-dimensional noise and scales linearly with dimension.

Contribution

It proposes a novel spectral filtering approach using Hessian eigenvalues to improve local intrinsic dimension estimation in high-dimensional spaces.

Findings

LHSD outperforms existing methods in robustness on synthetic and real data.

LHSD effectively detects memorization in large-scale diffusion models.

The method scales linearly with data dimension D.

Abstract

While diffusion models enable new approaches for estimating Local Intrinsic Dimension (LID), existing methods fail in high-dimensional spaces where noise from vast normal directions overwhelms the tangent signal. We propose Local Hessian Spectral Dimension (LHSD), which resolves this by applying spectral filtering to the log-density Hessian, explicitly cutting off large eigenvalues associated with normal directions to count zero-curvature tangent directions. Implemented using Stochastic Lanczos Quadrature (SLQ), LHSD avoids full Hessian construction, achieving linear scalability with dimension $D$. Experiments on synthetic and real data confirm LHSD's superior robustness and its utility in detecting memorization in large-scale diffusion models.