Empirical Bayes approach to Truth Discovery problems

TL;DR

This paper introduces an Empirical Bayes Estimator (EBE) that can enhance truth discovery algorithms by reducing expected error, especially when aggregating conflicting information from multiple sources.

Contribution

It formulates, proves, and empirically tests conditions under which EBE outperforms traditional weighted mean aggregation in truth discovery tasks.

Findings

EBE can dominate weighted mean under mild conditions.

EBE reduces expected error when used as a second step in TD algorithms.

Empirical tests confirm EBE's effectiveness in various scenarios.

Abstract

When aggregating information from conflicting sources, one's goal is to find the truth. Most real-value \emph{truth discovery} (TD) algorithms try to achieve this goal by estimating the competence of each source and then aggregating the conflicting information by weighing each source's answer proportionally to her competence. However, each of those algorithms requires more than a single source for such estimation and usually does not consider different estimation methods other than a weighted mean. Therefore, in this work we formulate, prove, and empirically test the conditions for an Empirical Bayes Estimator (EBE) to dominate the weighted mean aggregation. Our main result demonstrates that EBE, under mild conditions, can be used as a second step of any TD algorithm in order to reduce the expected error.