CaFA: Cost-aware, Feasible Attacks With Database Constraints Against Neural Tabular Classifiers

TL;DR

CaFA is a system that generates realistic adversarial examples for neural tabular classifiers by minimizing effort and satisfying database constraints, improving attack success and inconspicuousness.

Contribution

Introduces CaFA, a novel system combining cost-aware attack generation with automatically mined database constraints for more realistic adversarial examples.

Findings

CaFA outperforms prior methods in success rate and constraint satisfaction.

CaFA generates less noticeable adversarial perturbations.

Constraints used by CaFA are of higher quality than previous approaches.

Abstract

This work presents CaFA, a system for Cost-aware Feasible Attacks for assessing the robustness of neural tabular classifiers against adversarial examples realizable in the problem space, while minimizing adversaries' effort. To this end, CaFA leverages TabPGD$-$an algorithm we set forth to generate adversarial perturbations suitable for tabular data$-$ and incorporates integrity constraints automatically mined by state-of-the-art database methods. After producing adversarial examples in the feature space via TabPGD, CaFA projects them on the mined constraints, leading, in turn, to better attack realizability. We tested CaFA with three datasets and two architectures and found, among others, that the constraints we use are of higher quality (measured via soundness and completeness) than ones employed in prior work. Moreover, CaFA achieves higher feasible success rates$-$i.e., it generates adversarial examples that are often misclassified while satisfying constraints$-$than prior attacks while simultaneously perturbing few features with lower magnitudes, thus saving effort and improving inconspicuousness. We open-source CaFA, hoping it will serve as a generic system enabling machine-learning engineers to assess their models' robustness against realizable attacks, thus advancing deployed models' trustworthiness.