Low-Rank Adversarial PGD Attack
Dayana Savostianova, Emanuele Zangrando, Francesco Tudisco
TL;DR
This paper introduces a low-rank variation of the PGD adversarial attack that is faster and more memory-efficient, while maintaining or improving attack effectiveness on various models.
Contribution
We propose a novel low-rank PGD attack that leverages the spectral properties of perturbations, offering a more efficient alternative to traditional PGD.
Findings
Low-rank PGD often matches or exceeds the effectiveness of full-rank PGD.
The method requires significantly less memory and computational resources.
Low-rank PGD is suitable for adversarial training due to its efficiency.
Abstract
Adversarial attacks on deep neural network models have seen rapid development and are extensively used to study the stability of these networks. Among various adversarial strategies, Projected Gradient Descent (PGD) is a widely adopted method in computer vision due to its effectiveness and quick implementation, making it suitable for adversarial training. In this work, we observe that in many cases, the perturbations computed using PGD predominantly affect only a portion of the singular value spectrum of the original image, suggesting that these perturbations are approximately low-rank. Motivated by this observation, we propose a variation of PGD that efficiently computes a low-rank attack. We extensively validate our method on a range of standard models as well as robust models that have undergone adversarial training. Our analysis indicates that the proposed low-rank PGD can be…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
- The paper provides a clear and reproducible extension to adversarial training, with a reasonable motivation grounded in the empirical observation that adversarial perturbations often reside in low-dimensional manifolds. - The implementation is straightforward and compatible with standard PGD training, which could make the approach potentially useful if its effects were better analyzed and justified.
1. The paper does not thoroughly compare with recent improvements to PGD-based adversarial training. Without these baselines, the claimed advantages of LoRa-PGD are difficult to evaluate in context. 2. The reported improvements are generally small (often within 0.2–0.4% robust accuracy) and in some cases within noise margins. It is unclear whether these differences are statistically significant or reproducible across random seeds. 3. The results show inconsistent trends as the rank varies—hi
- The method is elegant and easy to understand. The authors impose a low-rank factorization in an elegant manner following the LoRa approach. - The authors include several ablations to further understand the behavior of the proposed attack and showcase preliminary results on adapting the proposed attack for adversarial-training experiments.
### **Marginality of empirical improvements** As the main concern of this paper, I found that most reported differences relative to PGD are extremely small (often in the third decimal place in Table 1). Differences like 0.826 –> 0.827 or 0.546 –> 0.547 are within typical experimental noise and should not be considered meaningful without statistical evidence. Indeed, related to this observation, there are no per-experiment standard deviations/confidence intervals, nor are there hypothesis tests t
This paper identifies a common yet frequently overlooked characteristic of adversarial perturbations—their low-rank nature. This finding significantly reduces the computational time and resource requirements for generating adversarial examples, thereby enhancing the practical deployability of such attacks. The experimental validation is thorough and well-designed.
1. The insight regarding the low-rank nature of adversarial perturbations is valuable. However, the design of the proposed LoRa-PGD method appears relatively straightforward, as it essentially decomposes the adversarial perturbation into two matrices U and V for solution. Could the authors elaborate on other noteworthy aspects of the method's design? 2. While inference time is a consideration, the primary bottleneck in deploying adversarial attacks lies in their white-box dependency. The applica
- Clear, simple idea: turns an empirical low-rank observation about PGD perturbations into a practical factorized attack. - Sensible theory lens: nuclear-norm budgeting and singular-spectrum analysis offer an intuitive explanation for why it works. - Balanced evaluation across attack and defense, with transparent compute accounting that makes results easy to interpret.
The weaknesses are mostly about the baselines and reported results: - Limited efficiency gains vs. PGD: Despite the claims, the performance/efficiency trade-off appears modest. Table 1 and especially Figure 2 suggest that when wall-clock time is accounted for, LoRa-PGD’s advantage over standard PGD is small. Tables 2 and 4 hint at gains in certain setups, but overall the time-normalized improvements look limited. - Missing recent PGD-derived baselines: The paper omits several works that build o
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Cryptographic Implementations and Security