Imputation for prediction: beware of diminishing returns

TL;DR

This study investigates the impact of imputation quality on predictive accuracy, revealing that advanced imputation methods often provide limited benefits when using expressive models or missingness indicators.

Contribution

It clarifies when investing in sophisticated imputation methods improves predictions, highlighting the limited gains in many practical scenarios.

Findings

Imputation accuracy is less critical with expressive models.

Using missingness indicators enhances prediction performance.

Improving imputation yields minor gains with powerful models on real data.

Abstract

Missing values are prevalent across various fields, posing challenges for training and deploying predictive models. In this context, imputation is a common practice, driven by the hope that accurate imputations will enhance predictions. However, recent theoretical and empirical studies indicate that simple constant imputation can be consistent and competitive. This empirical study aims at clarifying if and when investing in advanced imputation methods yields significantly better predictions. Relating imputation and predictive accuracies across combinations of imputation and predictive models on 19 datasets, we show that imputation accuracy matters less i) when using expressive models, ii) when incorporating missingness indicators as complementary inputs, iii) matters much more for generated linear outcomes than for real-data outcomes. Interestingly, we also show that the use of the missingness indicator is beneficial to the prediction performance, even in MCAR scenarios. Overall, on real-data with powerful models, improving imputation only has a minor effect on prediction performance. Thus, investing in better imputations for improved predictions often offers limited benefits.