Accelerating Adversarial Perturbation by 50% with Semi-backward Propagation

TL;DR

This paper introduces a method to speed up adversarial perturbation generation by 50% through semi-backward propagation, which computes only output gradients, maintaining effectiveness while reducing computation time.

Contribution

The paper proposes a semi-backward propagation technique that accelerates adversarial perturbation computation without sacrificing utility, achieving 1.5x overall speedup.

Findings

Achieves 2x acceleration in backward propagation

Maintains utility without performance drop

Overall 1.5x faster adversarial perturbation generation

Abstract

Adversarial perturbation plays a significant role in the field of adversarial robustness, which solves a maximization problem over the input data. We show that the backward propagation of such optimization can accelerate $2\times$ (and thus the overall optimization including the forward propagation can accelerate $1.5\times$), without any utility drop, if we only compute the output gradient but not the parameter gradient during the backward propagation.