Missing Data Imputation by Reducing Mutual Information with Rectified Flows

TL;DR

This paper presents a new iterative missing data imputation method that reduces mutual information between data and missingness, leveraging rectified flows and outperforming existing techniques on various datasets.

Contribution

Introduces a mutual information reduction framework for data imputation using rectified flows, unifying and improving upon existing methods.

Findings

Superior imputation accuracy on synthetic datasets.

Effective handling of real-world missing data scenarios.

Theoretical connection to existing imputation techniques.

Abstract

This paper introduces a novel iterative method for missing data imputation that sequentially reduces the mutual information between data and the corresponding missingness mask. Inspired by GAN-based approaches that train generators to decrease the predictability of missingness patterns, our method explicitly targets this reduction in mutual information. Specifically, our algorithm iteratively minimizes the KL divergence between the joint distribution of the imputed data and missingness mask, and the product of their marginals from the previous iteration. We show that the optimal imputation under this framework can be achieved by solving an ODE whose velocity field minimizes a rectified flow training objective. We further illustrate that some existing imputation techniques can be interpreted as approximate special cases of our mutual-information-reducing framework. Comprehensive experiments on synthetic and real-world datasets validate the efficacy of our proposed approach, demonstrating its superior imputation performance. Our implementation is available at https://github.com/yujhml/MIRI-Imputation.