Natural Geometry of Robust Data Attribution: From Convex Models to Deep Networks

TL;DR

This paper introduces a robust framework for data attribution that accounts for geometric perturbations, providing the first non-vacuous certified bounds for neural network attribution and improving stability through a novel Wasserstein metric.

Contribution

It develops a unified framework extending from convex models to deep networks, introducing the Natural Wasserstein metric to eliminate spectral amplification and stabilize attribution estimates.

Findings

Natural W-TRAK certifies 68.7 ext{ extbackslash}% of ranking pairs on CIFAR-10, outperforming Euclidean baselines.

Spectral amplification inflates Lipschitz bounds by over 10,000 times in deep networks.

Self-Influence effectively detects label noise with 0.970 AUROC.

Abstract

Data attribution methods identify which training examples are responsible for a model's predictions, but their sensitivity to distributional perturbations undermines practical reliability. We present a unified framework for certified robust attribution that extends from convex models to deep networks. For convex settings, we derive Wasserstein-Robust Influence Functions (W-RIF) with provable coverage guarantees. For deep networks, we demonstrate that Euclidean certification is rendered vacuous by spectral amplification -- a mechanism where the inherent ill-conditioning of deep representations inflates Lipschitz bounds by over $10{,}000\times$. This explains why standard TRAK scores, while accurate point estimates, are geometrically fragile: naive Euclidean robustness analysis yields 0\% certification. Our key contribution is the Natural Wasserstein metric, which measures perturbations in the geometry induced by the model's own feature covariance. This eliminates spectral amplification, reducing worst-case sensitivity by $76\times$ and stabilizing attribution estimates. On CIFAR-10 with ResNet-18, Natural W-TRAK certifies 68.7\% of ranking pairs compared to 0\% for Euclidean baselines -- to our knowledge, the first non-vacuous certified bounds for neural network attribution. Furthermore, we prove that the Self-Influence term arising from our analysis equals the Lipschitz constant governing attribution stability, providing theoretical grounding for leverage-based anomaly detection. Empirically, Self-Influence achieves 0.970 AUROC for label noise detection, identifying 94.1\% of corrupted labels by examining just the top 20\% of training data.