Identifying Adversary Characteristics from an Observed Attack

TL;DR

This paper introduces a domain-agnostic framework to identify the most probable attacker behind observed data-manipulation attacks on machine learning models, aiding in targeted defense strategies.

Contribution

It proposes a novel framework for attacker identification from observed attacks, addressing non-identifiability and improving defense effectiveness.

Findings

Framework effectively identifies likely attackers in various scenarios

Knowledge of attacker characteristics enhances defense performance

Demonstrated applicability across multiple learning models

Abstract

When used in automated decision-making systems, machine learning (ML) models are vulnerable to data-manipulation attacks. Some defense mechanisms (e.g., adversarial regularization) directly affect the ML models while others (e.g., anomaly detection) act within the broader system. In this paper we consider a different task for defending the adversary, focusing on the attacker, rather than the attack. We present and demonstrate a framework for identifying characteristics about the attacker from an observed attack. We prove that, without additional knowledge, the attacker is non-identifiable (multiple potential attackers would perform the same observed attack). To address this challenge, we propose a domain-agnostic framework to identify the most probable attacker. This framework aids the defender in two ways. First, knowledge about the attacker can be leveraged for exogenous mitigation (i.e., addressing the vulnerability by altering the decision-making system outside the learning algorithm and/or limiting the attacker's capability). Second, when implementing defense methods that directly affect the learning process (e.g., adversarial regularization), knowledge of the specific attacker improves performance. We present the details of our framework and illustrate its applicability through specific instantiations on a variety of learners.