M-DEW: Extending Dynamic Ensemble Weighting to Handle Missing Values

TL;DR

M-DEW introduces a novel AutoML approach that dynamically constructs and weights imputation-prediction pipelines during inference, effectively handling missing data to improve downstream task performance.

Contribution

It extends dynamic ensemble weighting to full imputation-prediction pipelines, optimizing the entire process jointly for better accuracy and calibration.

Findings

Outperforms state-of-the-art in model perplexity in 17/18 experiments.

Improves average precision in 13/18 experiments.

Reduces computational overhead compared to existing methods.

Abstract

Missing value imputation is a crucial preprocessing step for many machine learning problems. However, it is often considered as a separate subtask from downstream applications such as classification, regression, or clustering, and thus is not optimized together with them. We hypothesize that treating the imputation model and downstream task model together and optimizing over full pipelines will yield better results than treating them separately. Our work describes a novel AutoML technique for making downstream predictions with missing data that automatically handles preprocessing, model weighting, and selection during inference time, with minimal compute overhead. Specifically we develop M-DEW, a Dynamic missingness-aware Ensemble Weighting (DEW) approach, that constructs a set of two-stage imputation-prediction pipelines, trains each component separately, and dynamically calculates a set of pipeline weights for each sample during inference time. We thus extend previous work on dynamic ensemble weighting to handle missing data at the level of full imputation-prediction pipelines, improving performance and calibration on downstream machine learning tasks over standard model averaging techniques. M-DEW is shown to outperform the state-of-the-art in that it produces statistically significant reductions in model perplexity in 17 out of 18 experiments, while improving average precision in 13 out of 18 experiments.