Is Data Clustering in Adversarial Settings Secure?

TL;DR

This paper investigates the security of clustering algorithms in adversarial settings, proposing a framework to identify and evaluate potential attacks that can significantly poison clustering results.

Contribution

It introduces a general framework for analyzing attack strategies against clustering algorithms, including a case study on hierarchical clustering and real-world data.

Findings

Attacks can significantly poison clustering with minimal input manipulation.

Attack samples can be hidden within existing clusters.

Clustering security vulnerabilities depend on adversary's knowledge and capabilities.

Abstract

Clustering algorithms have been increasingly adopted in security applications to spot dangerous or illicit activities. However, they have not been originally devised to deal with deliberate attack attempts that may aim to subvert the clustering process itself. Whether clustering can be safely adopted in such settings remains thus questionable. In this work we propose a general framework that allows one to identify potential attacks against clustering algorithms, and to evaluate their impact, by making specific assumptions on the adversary's goal, knowledge of the attacked system, and capabilities of manipulating the input data. We show that an attacker may significantly poison the whole clustering process by adding a relatively small percentage of attack samples to the input data, and that some attack samples may be obfuscated to be hidden within some existing clusters. We present a case study on single-linkage hierarchical clustering, and report experiments on clustering of malware samples and handwritten digits.