Feature Importance Guided Attack: A Model Agnostic Adversarial Attack

TL;DR

This paper introduces FIGA, a model-agnostic adversarial attack method that perturbs feature importance in tabular datasets to evade detection models without requiring gradient information.

Contribution

The paper formalizes a feature importance guided attack algorithm for heterogeneous tabular data, demonstrating its effectiveness in evading various models and generating valid adversarial phishing examples.

Findings

Achieves 94% success rate on phishing detection models.

Successfully generates visually identical adversarial phishing sites.

Extends attack to real-world feasible perturbations in phishing domain.

Abstract

Research in adversarial learning has primarily focused on homogeneous unstructured datasets, which often map into the problem space naturally. Inverting a feature space attack on heterogeneous datasets into the problem space is much more challenging, particularly the task of finding the perturbation to perform. This work presents a formal search strategy: the `Feature Importance Guided Attack' (FIGA), which finds perturbations in the feature space of heterogeneous tabular datasets to produce evasion attacks. We first demonstrate FIGA in the feature space and then in the problem space. FIGA assumes no prior knowledge of the defending model's learning algorithm and does not require any gradient information. FIGA assumes knowledge of the feature representation and the mean feature values of defending model's dataset. FIGA leverages feature importance rankings by perturbing the most important features of the input in the direction of the target class. While FIGA is conceptually similar to other work which uses feature selection processes (e.g., mimicry attacks), we formalize an attack algorithm with three tunable parameters and investigate the strength of FIGA on tabular datasets. We demonstrate the effectiveness of FIGA by evading phishing detection models trained on four different tabular phishing datasets and one financial dataset with an average success rate of 94%. We extend FIGA to the phishing problem space by limiting the possible perturbations to be valid and feasible in the phishing domain. We generate valid adversarial phishing sites that are visually identical to their unperturbed counterpart and use them to attack six tabular ML models achieving a 13.05% average success rate.