Constrained Adaptive Attack: Effective Adversarial Attack Against Deep Neural Networks for Tabular Data

TL;DR

This paper introduces new adversarial attack methods, CAPGD and CAA, that significantly improve the ability to evaluate the robustness of deep neural networks for tabular data, revealing vulnerabilities and setting new standards for testing defenses.

Contribution

The paper proposes CAPGD, a gradient attack overcoming previous limitations, and CAA, a combined attack that outperforms existing methods in effectiveness and efficiency for tabular data models.

Findings

CAPGD degrades accuracy up to 81% points more than previous gradient attacks.

CAA outperforms all existing attacks in 17 of 20 settings, reducing accuracy by up to 96.1%.

The attacks are up to five times faster than the best existing search-based attack.

Abstract

State-of-the-art deep learning models for tabular data have recently achieved acceptable performance to be deployed in industrial settings. However, the robustness of these models remains scarcely explored. Contrary to computer vision, there are no effective attacks to properly evaluate the adversarial robustness of deep tabular models due to intrinsic properties of tabular data, such as categorical features, immutability, and feature relationship constraints. To fill this gap, we first propose CAPGD, a gradient attack that overcomes the failures of existing gradient attacks with adaptive mechanisms. This new attack does not require parameter tuning and further degrades the accuracy, up to 81% points compared to the previous gradient attacks. Second, we design CAA, an efficient evasion attack that combines our CAPGD attack and MOEVA, the best search-based attack. We demonstrate the effectiveness of our attacks on five architectures and four critical use cases. Our empirical study demonstrates that CAA outperforms all existing attacks in 17 over the 20 settings, and leads to a drop in the accuracy by up to 96.1% points and 21.9% points compared to CAPGD and MOEVA respectively while being up to five times faster than MOEVA. Given the effectiveness and efficiency of our new attacks, we argue that they should become the minimal test for any new defense or robust architectures in tabular machine learning.