# Adversarial Robustness vs Model Compression, or Both?

## TL;DR

This paper introduces a framework combining adversarial training with weight pruning to achieve model compression without sacrificing robustness, addressing the challenge of large model capacity requirements.

## Contribution

It proposes a novel concurrent adversarial training and pruning method that maintains robustness while reducing model size, and investigates the importance of pruning in adversarial settings.

## Key findings

- Pruning is essential for reducing model size in adversarial training.
- Small models trained from scratch cannot match robustness of larger models.
- The proposed method preserves robustness while enabling compression.

## Abstract

It is well known that deep neural networks (DNNs) are vulnerable to adversarial attacks, which are implemented by adding crafted perturbations onto benign examples. Min-max robust optimization based adversarial training can provide a notion of security against adversarial attacks. However, adversarial robustness requires a significantly larger capacity of the network than that for the natural training with only benign examples. This paper proposes a framework of concurrent adversarial training and weight pruning that enables model compression while still preserving the adversarial robustness and essentially tackles the dilemma of adversarial training. Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting, training a small model from scratch even with inherited initialization from the large model cannot achieve both adversarial robustness and high standard accuracy. Code is available at https://github.com/yeshaokai/Robustness-Aware-Pruning-ADMM.