TA3: Testing Against Adversarial Attacks on Machine Learning Models

TL;DR

This paper introduces TA3, an interactive system that incorporates human-in-the-loop techniques to test decision tree models against adversarial attacks, enhancing evaluation and understanding of model robustness.

Contribution

The paper presents the design of TA3, a novel interactive system that integrates human expertise into adversarial testing workflows for machine learning models.

Findings

HITL enhances attack simulation and impact evaluation.

TA3 effectively tests decision trees against One Pixel Attack.

Potential for extending HITL to other ML models and attacks.

Abstract

Adversarial attacks are major threats to the deployment of machine learning (ML) models in many applications. Testing ML models against such attacks is becoming an essential step for evaluating and improving ML models. In this paper, we report the design and development of an interactive system for aiding the workflow of Testing Against Adversarial Attacks (TA3). In particular, with TA3, human-in-the-loop (HITL) enables human-steered attack simulation and visualization-assisted attack impact evaluation. While the current version of TA3 focuses on testing decision tree models against adversarial attacks based on the One Pixel Attack Method, it demonstrates the importance of HITL in ML testing and the potential application of HITL to the ML testing workflows for other types of ML models and other types of adversarial attacks.