Exploring the Complexity of Real‐World Health Data Record Linkage—An Exemplary Study Linking Cancer Registry and Claims Data

TL;DR

This study explores the challenges of linking health data using quasi-identifiers and finds that machine learning improves the accuracy of record linkage.

Contribution

The study introduces informed linkage algorithms using gold standard links and compares machine learning methods for health data linkage.

Findings

Gradient boosting achieved the best performance with 77% precision and 81% recall.

33% of cancer registry patients could not be uniquely identified using quasi-identifiers.

Using unique identifiers from a subsample improves linkage quality for the entire dataset.

Abstract

Record linkage based on quasi‐identifiers remains an important approach as not every data source provides a comprehensive unique identifier. In this study, the reasons for the failure of a linkage based on quasi‐identifiers were examined. Furthermore, informed algorithms using information on gold standard links were developed to investigate the potentially achievable linkage quality based on quasi‐identifiers.

The study population includes patients from an antidiabetic cohort from German claims and colorectal cancer patients from two German cancer registries. Linkage algorithms were applied using information on gold standard links. Informed linkage algorithms based on deterministic linkage, logistic regression, random forests, gradient boosting, and neural networks were derived and compared. Descriptive analyses were performed to identify reasons for the failure of linkage, such as discrepancies between data sources.

A gradient boosting‐based linkage approach performed best, achieving a precision (positive predictive value) of 77%, a recall (sensitivity) of 81%, and an F*‐measure (combining precision and recall) of 64%. Of 641 patients in GePaRD, 8% were not uniquely identifiable using birth year, sex, area of residence, and year and quarter of diagnosis, whereas 33% of 42 817 cancer registry patients were not uniquely identifiable with these quasi‐identifiers.

Linkage of German claims and cancer registry data based on quasi‐identifiers does result in insufficient linkage quality since subjects cannot be uniquely identified. It is advisable to use unique identifiers from a subsample, if available, to derive informed linkage algorithms for the entire sample. In this case, the machine learning technique gradient boosting has been found to outperform other methods.

Figures