Reducing Uncertainty of Schema Matching via Crowdsourcing with Accuracy Rates

TL;DR

This paper presents methods to leverage crowdsourcing with accuracy rates to reduce uncertainty in schema matching, introducing adaptive algorithms for question management to optimize information gain within budget constraints.

Contribution

It introduces novel frameworks and algorithms for dynamically managing crowdsourced questions, incorporating worker accuracy to improve schema matching certainty.

Findings

Algorithms effectively reduce uncertainty in schema matching.

Adaptive question management improves efficiency and accuracy.

Validated through simulations and real-world implementation.

Abstract

Schema matching is a central challenge for data integration systems. Inspired by the popularity and the success of crowdsourcing platforms, we explore the use of crowdsourcing to reduce the uncertainty of schema matching. Since crowdsourcing platforms are most effective for simple questions, we assume that each Correspondence Correctness Question (CCQ) asks the crowd to decide whether a given correspondence should exist in the correct matching. Furthermore, members of a crowd may sometimes return incorrect answers with different probabilities. Accuracy rates of individual crowd workers are probabilities of returning correct answers which can be attributes of CCQs as well as evaluations of individual workers. We prove that uncertainty reduction equals to entropy of answers minus entropy of crowds and show how to obtain lower and upper bounds for it. We propose frameworks and efficient algorithms to dynamically manage the CCQs to maximize the uncertainty reduction within a limited budget of questions. We develop two novel approaches, namely `Single CCQ' and `Multiple CCQ', which adaptively select, publish and manage questions. We verify the value of our solutions with simulation and real implementation.