In Nonparametric and High-Dimensional Models, Bayesian Ignorability is an Informative Prior

TL;DR

This paper explores the implications of Bayesian ignorability in high-dimensional models with missing data, revealing that it often conflicts with realistic prior beliefs about selection bias and proposing ways to incorporate sensible priors.

Contribution

It demonstrates that Bayesian ignorability implies overly concentrated priors on selection bias in high-dimensional settings and offers methods to incorporate realistic prior beliefs.

Findings

Bayesian ignorability constrains the prior on selection bias to be near zero.

In high-dimensional linear models, ignorability affects the posterior distribution significantly.

Proposes modeling strategies for sensible priors on selection bias.

Abstract

In problems with large amounts of missing data one must model two distinct data generating processes: the outcome process which generates the response and the missing data mechanism which determines the data we observe. Under the ignorability condition of Rubin (1976), however, likelihood-based inference for the outcome process does not depend on the missing data mechanism so that only the former needs to be estimated; partially because of this simplification, ignorability is often used as a baseline assumption. We study the implications of Bayesian ignorability in the presence of high-dimensional nuisance parameters and argue that ignorability is typically incompatible with sensible prior beliefs about the amount of selection bias. We show that, for many problems, ignorability directly implies that the prior on the selection bias is tightly concentrated around zero. This is demonstrated on several models of practical interest, and the effect of ignorability on the posterior distribution is characterized for high-dimensional linear models with a ridge regression prior. We then show both how to build high-dimensional models which encode sensible beliefs about the selection bias and also show that under certain narrow circumstances ignorability is less problematic.