HICM: An approach towards Harmonizing Indian Census Migration data and its applications

TL;DR

This paper introduces HICM, a data-centric framework to harmonize and correct biases in Indian census migration data, improving its reliability for analysis and policy-making.

Contribution

HICM is a novel approach that explicitly identifies and mitigates measurement and representativeness biases in Indian migration data using statistical diagnostics.

Findings

Bias-aware data correction improves data consistency.

HICM enhances the reliability of temporal migration analysis.

The framework supports robust policy-relevant migration studies.

Abstract

Reliable analysis of migration is critically dependent on the quality and consistency of the underlying data. Indian migration data, primarily derived from decennial census records, are affected by systematic gaps arising from uneven coverage and measurement inconsistencies across states and time. This paper presents a data-centric framework, HICM, for harmonizing Indian census migration data recorded under the Indian census and correcting prominent sources of bias prior to downstream analyses. We explicitly identify two types of bias across three decades of migration data: measurement bias and representativeness bias. We propose to address these gaps through principled pre-processing, mitigation, and validation strategies grounded in statistical diagnostics. An empirical evaluation of harmonized Indian interstate migration data reveals that bias-aware data correction substantially improves the consistency in the structure of the data and enhances the reliability of subsequent temporal analysis results. By improving data quality through reproducible data imputation and smoothing, this work advances migration analytics and provides a robust foundation for policy-relevant longitudinal network analysis of Indian internal migration.