Eris: Measuring discord among multidimensional data sources

TL;DR

This paper introduces a metric called Eris for quantifying disagreement among multidimensional data sources, especially when ground truth is unavailable, to improve data trustworthiness and decision-making.

Contribution

It defines the concept of data concordance and proposes a discordance metric along with algebraic operators and database implementations for efficient measurement.

Findings

Efficient discordance measurement on COVID-19 and synthetic data

Algebraic operators with correctness guarantees for data alignment

Linear and quadratic programming approaches for implementation

Abstract

Data integration is a classical problem in databases, typically decomposed into schema matching, entity matching and data fusion. To solve the latter, it is mostly assumed that ground truth can be determined. However, in general, the data gathering processes in the different sources are imperfect and cannot provide an accurate merging of values. Thus, in the absence of ways to determine ground truth, it is important to at least quantify how far from being internally consistent a dataset is. Hence, we propose definitions of concordant data and define a discordance metric as a way of measuring disagreement to improve decision making based on trustworthiness. We define the discord measurement problem of numerical attributes in which given a set of uncertain raw observations or aggregate results (such as case/hospitalization/death data relevant to COVID-19) and information on the alignment of different conceptualizations of the same reality (e.g., granularities or units), we wish to assess whether the different sources are concordant, or if not, use the discordance metric to quantify how discordant they are. We also define a set of algebraic operators to describe the alignments of different data sources with correctness guarantees, together with two alternative relational database implementations that reduce the problem to linear or quadratic programming. These are evaluated against both COVID-19 and synthetic data, and our experimental results show that discordance measurement can be performed efficiently in realistic situations.