Impact of White-Box Adversarial Attacks on Convolutional Neural Networks

TL;DR

This paper thoroughly examines how various white-box adversarial attacks impact CNN performance across multiple datasets, highlighting vulnerabilities and emphasizing the need for robust defenses in critical applications.

Contribution

It provides a comprehensive analysis of the effects of multiple sophisticated white-box adversarial attacks on CNNs, including their impact on performance metrics and image quality measures.

Findings

Different attacks significantly increase error rates.

Iterative attacks are more effective than single-step attacks.

Certain image quality metrics correlate with classification performance.

Abstract

Autonomous vehicle navigation and healthcare diagnostics are among the many fields where the reliability and security of machine learning models for image data are critical. We conduct a comprehensive investigation into the susceptibility of Convolutional Neural Networks (CNNs), which are widely used for image data, to white-box adversarial attacks. We investigate the effects of various sophisticated attacks -- Fast Gradient Sign Method, Basic Iterative Method, Jacobian-based Saliency Map Attack, Carlini & Wagner, Projected Gradient Descent, and DeepFool -- on CNN performance metrics, (e.g., loss, accuracy), the differential efficacy of adversarial techniques in increasing error rates, the relationship between perceived image quality metrics (e.g., ERGAS, PSNR, SSIM, and SAM) and classification performance, and the comparative effectiveness of iterative versus single-step attacks. Using the MNIST, CIFAR-10, CIFAR-100, and Fashio_MNIST datasets, we explore the effect of different attacks on the CNNs performance metrics by varying the hyperparameters of CNNs. Our study provides insights into the robustness of CNNs against adversarial threats, pinpoints vulnerabilities, and underscores the urgent need for developing robust defense mechanisms to protect CNNs and ensuring their trustworthy deployment in real-world scenarios.