Security Evaluation of Pattern Classifiers under Attack

TL;DR

This paper introduces a framework for empirically evaluating the security of pattern classifiers against attacks, highlighting the importance of security-aware design in adversarial environments.

Contribution

It proposes a systematic framework for security evaluation of classifiers during design, extending classical methods to account for adversarial manipulation.

Findings

Security evaluation reveals vulnerabilities in classifiers.

Framework applied to three real-world applications.

Security-aware design improves classifier robustness.

Abstract

Pattern classification systems are commonly used in adversarial applications, like biometric authentication, network intrusion detection, and spam filtering, in which data can be purposely manipulated by humans to undermine their operation. As this adversarial scenario is not taken into account by classical design methods, pattern classification systems may exhibit vulnerabilities, whose exploitation may severely affect their performance, and consequently limit their practical utility. Extending pattern classification theory and design methods to adversarial settings is thus a novel and very relevant research direction, which has not yet been pursued in a systematic way. In this paper, we address one of the main open issues: evaluating at design phase the security of pattern classifiers, namely, the performance degradation under potential attacks they may incur during operation. We propose a framework for empirical evaluation of classifier security that formalizes and generalizes the main ideas proposed in the literature, and give examples of its use in three real applications. Reported results show that security evaluation can provide a more complete understanding of the classifier's behavior in adversarial environments, and lead to better design choices.