A capture-recapture hidden Markov model framework for register-based inference of population size and dynamics

TL;DR

This paper introduces a scalable hidden Markov model framework for register-based population inference, effectively handling errors and dynamics, demonstrated through Swedish population register data.

Contribution

It develops a novel capture-recapture hidden Markov model that accounts for false positives, false negatives, and individual heterogeneity, enabling inference on population size and dynamics.

Findings

Successfully applied to Swedish registers, revealing new population insights.

Handles multiple registers with complex error structures.

Provides scalable maximum likelihood inference with uncertainty quantification.

Abstract

Accurate inference on population dynamics, such as migration and changes in population size, is essential for policymaking, resource allocation and demographic research. Traditional censuses are expensive, infrequent and not timely, leading many countries to adopt register-based approaches to replace or complement them. A primary challenge is that such registers are incomplete: even when individuals are present, their activities may not generate records in specific registers, resulting in false negative observation error. Conversely, some registers arise from administrative or household-level processes, so that individuals may appear in registers despite being absent, leading to false positive observation error. Existing approaches often either rely on ad-hoc decisions that ignore one or both error types, offer inference on population snapshots but not dynamics, or are computationally too slow for practical use. We propose a scalable framework for inferring population size and dynamics from register data, building on Cormack-Jolly-Seber type capture-recapture models formulated as hidden Markov models. Inference is carried out using maximum likelihood estimation, with uncertainty quantified via the Bag of Little Bootstraps. The model accounts for temporary emigration, incorporates an arbitrary number of possibly interacting registers subject to both error types, and allows observation probabilities to vary with individual characteristics and unobservable heterogeneity. We illustrate the approach using Swedish population registers, where overcoverage - individuals registered as living in the country although they are no longer present - provides a motivating example. The application yields new insights into population dynamics and individual trajectories.