Data Poisoning Attacks and Defenses to Crowdsourcing Systems

TL;DR

This paper investigates the vulnerability of crowdsourcing systems to data poisoning attacks, demonstrating their effectiveness and proposing defenses to mitigate malicious influence on aggregated data quality.

Contribution

It introduces the first formulation of data poisoning attacks in crowdsourcing as an optimization problem and proposes defenses to counteract these attacks.

Findings

Attacks significantly increase data estimation errors.

Proposed defenses effectively reduce attack impact.

Empirical validation on real-world datasets confirms results.

Abstract

A key challenge of big data analytics is how to collect a large volume of (labeled) data. Crowdsourcing aims to address this challenge via aggregating and estimating high-quality data (e.g., sentiment label for text) from pervasive clients/users. Existing studies on crowdsourcing focus on designing new methods to improve the aggregated data quality from unreliable/noisy clients. However, the security aspects of such crowdsourcing systems remain under-explored to date. We aim to bridge this gap in this work. Specifically, we show that crowdsourcing is vulnerable to data poisoning attacks, in which malicious clients provide carefully crafted data to corrupt the aggregated data. We formulate our proposed data poisoning attacks as an optimization problem that maximizes the error of the aggregated data. Our evaluation results on one synthetic and two real-world benchmark datasets demonstrate that the proposed attacks can substantially increase the estimation errors of the aggregated data. We also propose two defenses to reduce the impact of malicious clients. Our empirical results show that the proposed defenses can substantially reduce the estimation errors of the data poisoning attacks.