Amortized Variational Inference for Logistic Regression with Missing Covariates

TL;DR

This paper introduces AV-LR, an end-to-end amortized variational inference framework for logistic regression with missing data, offering comparable accuracy to traditional methods but with lower computational costs and flexibility for missing-not-at-random scenarios.

Contribution

AV-LR is a novel unified inference framework that directly models missing data in logistic regression using a simple inference network, improving efficiency and extending to complex missingness mechanisms.

Findings

Achieves estimation accuracy comparable or better than EM-based methods.

Significantly reduces computational cost compared to traditional approaches.

Effectively handles missing-not-at-random data by modeling missingness mechanism.

Abstract

Missing covariate data pose a significant challenge to statistical inference and machine learning, particularly for classification tasks like logistic regression. Classical iterative approaches (EM, multiple imputation) are often computationally intensive, sensitive to high missingness rates, and limited in uncertainty propagation. Recent deep generative models based on VAEs show promise but rely on complex latent representations. We propose Amortized Variational Inference for Logistic Regression (AV-LR), a unified end-to-end framework for binary logistic regression with missing covariates. AV-LR integrates a probabilistic generative model with a simple amortized inference network, trained jointly by maximizing the evidence lower bound. Unlike competing methods, AV-LR performs inference directly in the space of missing data without additional latent variables, using a single inference network and a linear layer that jointly estimate regression parameters and the missingness mechanism. AV-LR achieves estimation accuracy comparable to or better than state-of-the-art EM-like algorithms, with significantly lower computational cost. It naturally extends to missing-not-at-random settings by explicitly modeling the missingness mechanism. Empirical results on synthetic and real-world datasets confirm its effectiveness and efficiency across various missing-data scenarios.