Decision Tree Design for Classification in Crowdsourcing Systems

TL;DR

This paper introduces a new sequential decision tree approach for crowdsourcing classification that accounts for worker unreliability, aiming to minimize misclassification probability through novel algorithms and worker assignment strategies.

Contribution

It proposes two algorithms for decision tree design in crowdsourcing, considering worker errors and optimizing the trade-off between cost and accuracy.

Findings

Algorithms effectively reduce misclassification probability.

Worker assignment strategies improve cost-performance balance.

Numerical results validate the proposed methods.

Abstract

In this paper, we present a novel sequential paradigm for classification in crowdsourcing systems. Considering that workers are unreliable and they perform the tests with errors, we study the construction of decision trees so as to minimize the probability of mis-classification. By exploiting the connection between probability of mis-classification and entropy at each level of the decision tree, we propose two algorithms for decision tree design. Furthermore, the worker assignment problem is studied when workers can be assigned to different tests of the decision tree to provide a trade-off between classification cost and resulting error performance. Numerical results are presented for illustration.