S2AP: Score-space Sharpness Minimization for Adversarial Pruning
Giorgio Piras, Qi Zhao, Fabio Brau, Maura Pintor, Christian Wressnegger, Battista Biggio
TL;DR
This paper introduces S2AP, a novel method that minimizes score-space sharpness during adversarial pruning, leading to more stable mask selection and improved robustness of pruned neural networks against adversarial attacks.
Contribution
The paper proposes a new score-space sharpness minimization technique for adversarial pruning, enhancing robustness and stability over existing methods.
Findings
S2AP reduces sharpness in score space effectively.
S2AP improves robustness across various datasets and models.
S2AP stabilizes mask selection during pruning.
Abstract
Adversarial pruning methods have emerged as a powerful tool for compressing neural networks while preserving robustness against adversarial attacks. These methods typically follow a three-step pipeline: (i) pretrain a robust model, (ii) select a binary mask for weight pruning, and (iii) finetune the pruned model. To select the binary mask, these methods minimize a robust loss by assigning an importance score to each weight, and then keep the weights with the highest scores. However, this score-space optimization can lead to sharp local minima in the robust loss landscape and, in turn, to an unstable mask selection, reducing the robustness of adversarial pruning methods. To overcome this issue, we propose a novel plug-in method for adversarial pruning, termed Score-space Sharpness-aware Adversarial Pruning (S2AP). Through our method, we introduce the concept of score-space sharpness…
Peer Reviews
Decision·Submitted to ICLR 2026
1. The paper identifies a clear and important issue, mask instability due to sharp score-space transitions. It provides empirical evidence of its impact on robustness. 2. Experiments span multiple datasets, models, and pruning methods, with consistent improvements in robust accuracy and mask stability. 3. S2AP is designed to integrate with existing pruning pipelines, making it practically useful.
My concerns are listed in following aspects. 1. The overall idea appears to be incremental, which is a direct extension of sharpness-aware minimization in weight-space. Applying this to score-space is intuitive but not conceptually novel. Similar ideas have already been explored in AdaSAP and S2-SAM, which also target sharpness minimization for sparse training. 2. The min–max formulation is plausible but lacks rigorous analysis. There is no convergence proof, no bounds on sharpness reduction,
1. The idea of transferring “sharpness minimization” from the parameter space to the score space for mask optimization is novel. 2. S2AP is a plug-in module that can be integrated into any score-based pruning framework without modifying the original objective. 3. Extensive experiments demonstrate S2AP’s generality and robustness across different scenarios. 4. They propose a “mask stability” metric to quantitatively verify that S2AP makes the mask search process smoother and more stable.
1. The paper contains several formatting problems, including incorrect citation format (e.g, line 43) and missing punctuation (e.g. line 245). 2. Limited comparison with existing sharpness-aware methods. While the authors acknowledge that S2AP is inspired by sharpness-aware approaches such as SAM [1] and AWP [2], no experiments compare S2AP directly with SAM or AWP under the same conditions. As a result, it remains unclear how much the proposed method truly improves adversarial robustness compar
1. The work extends the concept of sharpness minimization from the parameter space to the score-space setting. 2. It proposes a score-space sharpness minimization approach tailored for adversarial pruning. 3. The proposed method improves adversarial robustness and outperforms existing pruning baselines such as HARP and HYDRA.
1. The motivation is somewhat weak, and key concepts such as the sensitivity of top-k selection to score variations and the link between score-space sharpness and robustness are insufficiently explained. 2. The fairness of comparison may be questionable since it is unclear whether HARP and HYDRA are fine-tuned with AWP as in S2AP. Results under identical fine-tuning setups should be provided. Minor： 3. The explanation of “trade-off in generalization” and its connection with ∆ is unclear. 4.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Advanced Neural Network Applications · Stochastic Gradient Optimization Techniques