Correcting for Nonignorable Nonresponse Bias in Ordinal Observational Survey Data

TL;DR

This paper introduces a practical method to correct nonignorable nonresponse bias in ordinal survey data by leveraging response-propensity proxies, improving the accuracy of survey estimates especially for variables like life satisfaction.

Contribution

It generalizes the VRP framework to ordinal outcomes and provides a maximum likelihood estimator implemented in R for bias correction in surveys.

Findings

Corrects bias in ordinal survey data using response-propensity proxies.

Significant impact on life satisfaction estimates, negligible on economic evaluations.

Method validated with the 2024 ANES data.

Abstract

Many political surveys rely on post-stratification, raking, or related weighting adjustments to align respondents with the target population. But when respondents differ from nonrespondents on the outcome itself (nonignorable nonresponse), these adjustments can fail, introducing bias even into basic descriptives.We provide a practical method that corrects for nonignorable nonresponse by leveraging response-propensity proxies (e.g., interviewer-coded cooperativeness) observed among respondents to extrapolate toward nonrespondents, while directly integrating observable covariates and retaining the benefits of post-stratification with known population shares. The method generalizes the variable-response-propensity (VRP) framework of Peress (2010) from binary to ordinal outcomes, which are widely used to measure trust, satisfaction, and policy attitudes. The resulting estimator is computed by maximum likelihood and implemented in a compact R routine that handles both ordinal and binary outcomes. Using the 2024 American National Election Study (ANES), we show that accounting for nonignorable nonresponse produces substantively meaningful shifts for life satisfaction (estimated latent correlation $\rho \approx 0.49$), while yielding negligible changes for retrospective economic evaluations ($\rho \approx 0$), highlighting when nonignorable nonresponse substantively affects survey estimates.