Adversarial Transferability in Deep Denoising Models: Theoretical Insights and Robustness Enhancement via Out-of-Distribution Typical Set Sampling

TL;DR

This paper investigates why deep denoising models are vulnerable to adversarial attacks and proposes a novel training strategy, TS, that improves robustness by sampling from the out-of-distribution typical set.

Contribution

It provides a theoretical analysis of adversarial transferability in denoising models and introduces the TS training method to enhance robustness.

Findings

Adversarial samples deviate from the typical set, causing model failures.

The TS training method significantly improves robustness against adversarial attacks.

The proposed approach marginally improves denoising performance.

Abstract

Deep learning-based image denoising models demonstrate remarkable performance, but their lack of robustness analysis remains a significant concern. A major issue is that these models are susceptible to adversarial attacks, where small, carefully crafted perturbations to input data can cause them to fail. Surprisingly, perturbations specifically crafted for one model can easily transfer across various models, including CNNs, Transformers, unfolding models, and plug-and-play models, leading to failures in those models as well. Such high adversarial transferability is not observed in classification models. We analyze the possible underlying reasons behind the high adversarial transferability through a series of hypotheses and validation experiments. By characterizing the manifolds of Gaussian noise and adversarial perturbations using the concept of typical set and the asymptotic equipartition property, we prove that adversarial samples deviate slightly from the typical set of the original input distribution, causing the models to fail. Based on these insights, we propose a novel adversarial defense method: the Out-of-Distribution Typical Set Sampling Training strategy (TS). TS not only significantly enhances the model's robustness but also marginally improves denoising performance compared to the original model.