TL;DR
This paper introduces an un-pruning method for sparse models to better achieve machine unlearning, ensuring models forget deleted data effectively, with theoretical guarantees and new evaluation metrics.
Contribution
It proposes a novel un-pruning algorithm that integrates with existing unlearning methods, applicable to both structured and unstructured sparse models, with theoretical error bounds.
Findings
Un-pruning impacts the pruned topology based on deleted data.
Membership Inference Attack accuracy is unreliable for measuring forgetting.
New metrics are proposed to evaluate un-pruning success.
Abstract
Machine unlearning aims to efficiently eliminate the memory about deleted data from trained models and address the right to be forgotten. Despite the success of existing unlearning algorithms, unlearning in sparse models has not yet been well studied. In this paper, we empirically find that the deleted data has an impact on the pruned topology in a sparse model. Motivated by the observation and the right to be forgotten, we define a new terminology ``un-pruning" to eliminate the impact of deleted data on model pruning. Then we propose an un-pruning algorithm to approximate the pruned topology driven by retained data. We remark that any existing unlearning algorithm can be integrated with the proposed un-pruning workflow and the error of un-pruning is upper-bounded in theory. Also, our un-pruning algorithm can be applied to both structured sparse models and unstructured sparse models. In…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
1. The discovery of the pruning mask's data dependency is meaningful, and it inspires future work. The intuition and theoretical analysis of updating pruning mask is sound. 2. The proposed method is plug-in-and-play (or can be seen as a regularization), so it can be used to enhance existing unlearning methods. 3. This paper challenges MIA with plausible reasoning, and proposes their alternative. I am somewhat convinced by Section 5, though I think we can still refer to MIA with some aid metric
1. I am confused by the retrained+repruned goal, why is it the "gold standard"? In conventional unlearning literature, including sparsity-based unlearning work, e.g., [1], the gold standard is mostly the retrained model. Previous sparsity-inspired methods adopt sparsification to approach the retrained model, e.g. zeroing out parameters that can be activated by forget samples, or in other words parameters not important to retain samples and model performance. If the method only targets pruned, re
- The paper introduces a problem that, at least to me, appears novel and unexplored: unlearning in sparse models. - The paper integrates a suitable range of existing unlearning algorithms within the proposed un-pruning framework, which helps demonstrate the generality of the approach.
- The presentation quality could be significantly improved, as the current form makes the paper challenging to read and interpret. Additional comments are provided below. - The section: Analysis of Un-Pruning is very difficult to follow due to numerous typos or errors (I listed some of them below) and unclear explanations. The discussion of the differentiable pruning strategy is confusing, as the actual algorithm relies on hard, discrete mask updates based on magnitude thresholding. The proposed
1. The manuscript proposes a new research question: how to eliminate the impact of deleted data on sparse model pruning topology. 2. The manuscript proposes an "un-pruning" algorithm that can approximate pruning topologies driven by retained data without the need for expensive re-training and re-pruning. The algorithm can integrate with any existing unloading algorithm and is suitable for both unstructured and structured topologies. 3. The manuscript points the vulnerability of MIA as an unload
1. The un-pruning strategy is mainly suitable for unstructured pruning algorithms that do not introduce additional learnable parameters for pruning. It is not used for structured pruning methods that use extra parameters to learn the pruning structure. 2. The legends are too small, which affects readability, for example in Fig 6. 3. The proposed Un-Pruning algorithm densifies the model, then applies the forgetting algorithm on this dense model, and finally prunes the weights. The authors do not
- The setup of unpruning is interesting. - The overall problem has potential to be relevant.
**Conceptual clarity and motivation:** - The term *un-pruning* is a bit incorrect and imprecise. I understand the authors' idea of unifying unlearning and pruning, but un-pruning truly deviates from what the paper is presenting, and I do not think reflects “semantically” what is done in this paper. This could be more simply described as un-learning under a dynamic sparsity constraint. For what concerns naming, in addition, I do not honestly understand the point of using terms retraining and re
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
