Spectral Graph Sparsification Preserves Representation Geometry in Graph Neural Networks

TL;DR

This paper demonstrates that spectral graph sparsification maintains the geometric structure of GNN embeddings and ensures stability in training, supported by theoretical proofs and empirical validation.

Contribution

It provides the first theoretical and empirical analysis showing spectral sparsification preserves embedding geometry and training stability in GNNs.

Findings

Spectral sparsification induces only $O(psilon)$ perturbations in polynomial graph filters and representations.

Embedding geometry, including Gram matrices and class means, remains stable under spectral sparsification.

Training dynamics on sparse graphs closely match those on dense graphs, indicating stability and robustness.

Abstract

Spectral graph sparsification is a classical tool for reducing graph complexity while preserving Laplacian quadratic forms. In graph neural networks (GNNs), sparsification is often used to accelerate computation while maintaining predictive performance. In this work, we study a complementary representation-level question: does sparsification preserve the geometry of learned embeddings? For polynomial-filter GNNs, we prove that any $\epsilon$-spectral sparsifier induces $O(\epsilon)$ perturbations in polynomial graph filters, multilayer hidden representations, and their Gram matrices. These guarantees imply stability of squared pairwise distances, class means, and covariance structure in embedding space. We further establish finite-time training stability: under smoothness and boundedness assumptions, gradient descent on dense and sparsified graphs produces weight trajectories whose separation grows at most proportionally to the sparsification distortion. Empirically, effective-resistance sparsification validates the predicted perturbation chain on synthetic graphs and preserves hidden representation geometry on real datasets. In our experiments, the gram matrix and training dynamics show low divergence even under substantial sparsification, consistent with the predicted stability under spectral sparsification. Hidden Gram preservation strongly predicts neighborhood preservation and class-centroid stability across FashionMNIST, Cora, and Paul15. Together, these results show that spectral sparsification preserves not only graph operators, but also the representation geometry that supports downstream use of GNN embeddings for interpretability.