TopoPrune: Robust Data Pruning via Unified Latent Space Topology

TL;DR

Topological data pruning with TopoPrune enhances robustness and transferability by capturing intrinsic data structure through a dual-scale topological approach, improving stability and performance in model pruning tasks.

Contribution

Introduces TopoPrune, a novel topology-based framework that stabilizes data pruning by leveraging intrinsic data structure via persistent homology and manifold approximation.

Findings

Achieves high accuracy at 90% dataset pruning rates.

Demonstrates robustness to latent feature noise perturbations.

Shows superior transferability across different neural network architectures.

Abstract

Geometric data pruning methods, while practical for leveraging pretrained models, are fundamentally unstable. Their reliance on extrinsic geometry renders them highly sensitive to latent space perturbations, causing performance to degrade during cross-architecture transfer or in the presence of feature noise. We introduce TopoPrune, a framework which resolves this challenge by leveraging topology to capture the stable, intrinsic structure of data. TopoPrune operates at two scales, (1) utilizing a topology-aware manifold approximation to establish a global low-dimensional embedding of the dataset. Subsequently, (2) it employs differentiable persistent homology to perform a local topological optimization on the manifold embeddings, ranking samples by their structural complexity. We demonstrate that our unified dual-scale topological approach ensures high accuracy and precision, particularly at significant dataset pruning rates (e.g., 90%). Furthermore, through the inherent stability properties of topology, TopoPrune is (a) exceptionally robust to noise perturbations of latent feature embeddings and (b) demonstrates superior transferability across diverse network architectures. This study demonstrates a promising avenue towards stable and principled topology-based frameworks for robust data-efficient learning.