Training Deep Normalizing Flow Models in Highly Incomplete Data Scenarios with Prior Regularization

TL;DR

This paper introduces a novel prior regularization framework for training deep normalizing flow models on highly incomplete data, significantly improving performance in high missing data scenarios.

Contribution

It proposes a joint optimization approach with prior regularization to effectively learn data distributions from highly incomplete datasets, outperforming existing methods.

Findings

Outperforms competing techniques in high missing data regimes

Enables stable training of normalizing flows with up to 90% missing data

Demonstrates robustness across various incomplete data scenarios

Abstract

Deep generative frameworks including GANs and normalizing flow models have proven successful at filling in missing values in partially observed data samples by effectively learning -- either explicitly or implicitly -- complex, high-dimensional statistical distributions. In tasks where the data available for learning is only partially observed, however, their performance decays monotonically as a function of the data missingness rate. In high missing data rate regimes (e.g., 60% and above), it has been observed that state-of-the-art models tend to break down and produce unrealistic and/or semantically inaccurate data. We propose a novel framework to facilitate the learning of data distributions in high paucity scenarios that is inspired by traditional formulations of solutions to ill-posed problems. The proposed framework naturally stems from posing the process of learning from incomplete data as a joint optimization task of the parameters of the model being learned and the missing data values. The method involves enforcing a prior regularization term that seamlessly integrates with objectives used to train explicit and tractable deep generative frameworks such as deep normalizing flow models. We demonstrate via extensive experimental validation that the proposed framework outperforms competing techniques, particularly as the rate of data paucity approaches unity.