Survival analysis under imperfect record linkage using historic census data

TL;DR

This paper explores how to handle missing survival data from imperfectly linked census records and finds that imputation based on conditional survival is more effective for analyzing mortality risks.

Contribution

The paper introduces and evaluates modified imputation methods for handling missing survival times due to imperfect record linkage in historical cohorts.

Findings

Conditional survival-based imputation reduces bias and increases efficiency in estimating hazard ratios and median survival times.

Occupational asbestos exposure is significantly linked to higher mortality, especially among Black individuals and males.

Imputation methods' performance depends on the missingness mechanism and the parameter being estimated.

Abstract

Advancements in linking publicly available census records with vital and administrative records have enabled novel investigations in epidemiology and social history. However, in the absence of unique identifiers, the linkage of the records may be uncertain or only be successful for a subset of the census cohort, resulting in missing data. For survival analysis, differential ascertainment of event times can impact inference on risk associations and median survival.

We modify some existing approaches that are commonly used to handle missing survival times to accommodate this imperfect linkage situation including complete case analysis, censoring, weighting, and several multiple imputation methods. We then conduct simulation studies to compare the performance of the proposed approaches in estimating the associations of a risk factor or exposure in terms of hazard ratio (HR) and median survival times in the presence of missing survival times. The effects of different missing data mechanisms and exposure-survival associations on their performance are also explored. The approaches are applied to a historic cohort of residents in Ambler, PA, established using the 1930 US census, from which only 2,440 out of 4,514 individuals (54%) had death records retrievable from publicly available data sources and death certificates. Using this cohort, we examine the effects of occupational and paraoccupational asbestos exposure on survival and disparities in mortality by race and gender.

We show that imputation based on conditional survival results in less bias and greater efficiency relative to a complete case analysis when estimating log-hazard ratios and median survival times. When the approaches are applied to the Ambler cohort, we find a significant association between occupational exposure and mortality, particularly among black individuals and males, but not between paraoccupational exposure and mortality.

This investigation illustrates the strengths and weaknesses of different imputation methods for missing survival times due to imperfect linkage of the administrative or registry data. The performance of the methods may depend on the missingness process as well as the parameter being estimated and models of interest, and such factors should be considered when choosing the methods to address the missing event times.

The online version contains supplementary material available at 10.1186/s12874-024-02194-6.

Figures