T-Crowd: Effective Crowdsourcing for Tabular Data

TL;DR

T-Crowd is a system that improves crowdsourcing for tabular data by considering attribute relationships and worker trustworthiness, leading to faster convergence and better accuracy in data cleaning and analysis.

Contribution

It introduces a novel approach that models attribute relationships and worker trustworthiness to enhance crowdsourcing efficiency and accuracy for tabular data.

Findings

Outperforms existing methods in truth inference accuracy.

Reduces crowdsourcing costs significantly.

Supports both categorical and continuous data types.

Abstract

Crowdsourcing employs human workers to solve computer-hard problems, such as data cleaning, entity resolution, and sentiment analysis. When crowdsourcing tabular data, e.g., the attribute values of an entity set, a worker's answers on the different attributes (e.g., the nationality and age of a celebrity star) are often treated independently. This assumption is not always true and can lead to suboptimal crowdsourcing performance. In this paper, we present the T-Crowd system, which takes into consideration the intricate relationships among tasks, in order to converge faster to their true values. Particularly, T-Crowd integrates each worker's answers on different attributes to effectively learn his/her trustworthiness and the true data values. The attribute relationship information is also used to guide task allocation to workers. Finally, T-Crowd seamlessly supports categorical and continuous attributes, which are the two main datatypes found in typical databases. Our extensive experiments on real and synthetic datasets show that T-Crowd outperforms state-of-the-art methods in terms of truth inference and reducing the cost of crowdsourcing.