Prediction Models That Learn to Avoid Missing Values

TL;DR

This paper introduces a framework called missingness-avoiding (MA) learning that trains models to minimize reliance on missing features at test time, enhancing interpretability and maintaining accuracy.

Contribution

The paper develops tailored MA algorithms for decision trees, linear models, and ensembles, incorporating regularization to reduce dependence on missing features.

Findings

MA models effectively reduce reliance on missing features

MA models maintain competitive predictive performance

Framework enhances interpretability with missing data

Abstract

Handling missing values at test time is challenging for machine learning models, especially when aiming for both high accuracy and interpretability. Established approaches often add bias through imputation or excessive model complexity via missingness indicators. Moreover, either method can obscure interpretability, making it harder to understand how the model utilizes the observed variables in predictions. We propose missingness-avoiding (MA) machine learning, a general framework for training models to rarely require the values of missing (or imputed) features at test time. We create tailored MA learning algorithms for decision trees, tree ensembles, and sparse linear models by incorporating classifier-specific regularization terms in their learning objectives. The tree-based models leverage contextual missingness by reducing reliance on missing values based on the observed context. Experiments on real-world datasets demonstrate that MA-DT, MA-LASSO, MA-RF, and MA-GBT effectively reduce the reliance on features with missing values while maintaining predictive performance competitive with their unregularized counterparts. This shows that our framework gives practitioners a powerful tool to maintain interpretability in predictions with test-time missing values.