Crowdsourced Truth Discovery in the Presence of Hierarchies for Knowledge Fusion

TL;DR

This paper introduces a probabilistic model for truth discovery that accounts for hierarchical structures in data and leverages crowdsourcing to improve accuracy in knowledge fusion from unstructured sources.

Contribution

It presents a novel hierarchical-aware truth discovery model combined with a crowdsourcing task assignment algorithm, enhancing accuracy in knowledge fusion tasks.

Findings

Effective truth inference with hierarchical data structures

Crowdsourcing improves the accuracy of unstructured data claims

Proposed algorithms outperform baseline methods in real datasets

Abstract

Existing works for truth discovery in categorical data usually assume that claimed values are mutually exclusive and only one among them is correct. However, many claimed values are not mutually exclusive even for functional predicates due to their hierarchical structures. Thus, we need to consider the hierarchical structure to effectively estimate the trustworthiness of the sources and infer the truths. We propose a probabilistic model to utilize the hierarchical structures and an inference algorithm to find the truths. In addition, in the knowledge fusion, the step of automatically extracting information from unstructured data (e.g., text) generates a lot of false claims. To take advantages of the human cognitive abilities in understanding unstructured data, we utilize crowdsourcing to refine the result of the truth discovery. We propose a task assignment algorithm to maximize the accuracy of the inferred truths. The performance study with real-life datasets confirms the effectiveness of our truth inference and task assignment algorithms.