A Practical Approach to Proper Inference with Linked Data

TL;DR

This paper introduces scalable Bayesian methods for canonicalization in entity resolution, improving the propagation of uncertainty to downstream inference tasks like regression, demonstrated on simulated and real voter data.

Contribution

It proposes five scalable Bayesian canonicalization methods that effectively incorporate ER uncertainty into downstream analysis, enhancing inference accuracy.

Findings

Bayesian canonicalization improves downstream inference accuracy.

Methods are scalable and applicable to general data scenarios.

Empirical evaluation shows better prediction and coverage in regression tasks.

Abstract

Entity resolution (ER), comprising record linkage and de-duplication, is the process of merging noisy databases in the absence of unique identifiers to remove duplicate entities. One major challenge of analysis with linked data is identifying a representative record among determined matches to pass to an inferential or predictive task, referred to as the \emph{downstream task}. Additionally, incorporating uncertainty from ER in the downstream task is critical to ensure proper inference. To bridge the gap between ER and the downstream task in an analysis pipeline, we propose five methods to choose a representative (or canonical) record from linked data, referred to as canonicalization. Our methods are scalable in the number of records, appropriate in general data scenarios, and provide natural error propagation via a Bayesian canonicalization stage. The proposed methodology is evaluated on three simulated data sets and one application -- determining the relationship between demographic information and party affiliation in voter registration data from the North Carolina State Board of Elections. We first perform Bayesian ER and evaluate our proposed methods for canonicalization before considering the downstream tasks of linear and logistic regression. Bayesian canonicalization methods are empirically shown to improve downstream inference in both settings through prediction and coverage.