# Graph Mining Meets Crowdsourcing: Extracting Experts for Answer Aggregation

## TL;DR

This paper introduces a graph-mining approach to identify reliable experts within crowdsourcing tasks, enhancing answer aggregation accuracy especially when non-experts dominate, and demonstrates improved performance over existing methods.

## Contribution

It proposes the novel concept of 'expert core' and efficient algorithms for extracting it, improving answer aggregation in crowdsourcing by leveraging expert identification.

## Key findings

- Expert core extraction improves answer accuracy.
- Algorithms outperform state-of-the-art methods.
- Theoretical justification supports the approach.

## Abstract

Aggregating responses from crowd workers is a fundamental task in the process of crowdsourcing. In cases where a few experts are overwhelmed by a large number of non-experts, most answer aggregation algorithms such as the majority voting fail to identify the correct answers. Therefore, it is crucial to extract reliable experts from the crowd workers. In this study, we introduce the notion of "expert core", which is a set of workers that is very unlikely to contain a non-expert. We design a graph-mining-based efficient algorithm that exactly computes the expert core. To answer the aggregation task, we propose two types of algorithms. The first one incorporates the expert core into existing answer aggregation algorithms such as the majority voting, whereas the second one utilizes information provided by the expert core extraction algorithm pertaining to the reliability of workers. We then give a theoretical justification for the first type of algorithm. Computational experiments using synthetic and real-world datasets demonstrate that our proposed answer aggregation algorithms outperform state-of-the-art algorithms.