On Security and Sparsity of Linear Classifiers for Adversarial Settings

TL;DR

This paper investigates the security vulnerabilities of linear classifiers in adversarial settings, analyzing the impact of regularization and sparsity, and proposes a novel regularizer to enhance both security and efficiency.

Contribution

It introduces a new octagonal regularizer that balances security and sparsity in linear classifiers under adversarial attacks.

Findings

The octagonal regularizer improves classifier robustness against evasion attacks.

Sparsity of feature weights correlates with increased security.

Empirical results demonstrate enhanced performance in spam and malware detection.

Abstract

Machine-learning techniques are widely used in security-related applications, like spam and malware detection. However, in such settings, they have been shown to be vulnerable to adversarial attacks, including the deliberate manipulation of data at test time to evade detection. In this work, we focus on the vulnerability of linear classifiers to evasion attacks. This can be considered a relevant problem, as linear classifiers have been increasingly used in embedded systems and mobile devices for their low processing time and memory requirements. We exploit recent findings in robust optimization to investigate the link between regularization and security of linear classifiers, depending on the type of attack. We also analyze the relationship between the sparsity of feature weights, which is desirable for reducing processing cost, and the security of linear classifiers. We further propose a novel octagonal regularizer that allows us to achieve a proper trade-off between them. Finally, we empirically show how this regularizer can improve classifier security and sparsity in real-world application examples including spam and malware detection.