Diffusion Models for Tabular Data Imputation and Synthetic Data Generation

TL;DR

This paper introduces a novel diffusion model with transformer-based conditioning and dynamic masking for effective tabular data imputation and synthetic data generation, outperforming existing methods in efficiency, similarity, and privacy.

Contribution

It presents a new diffusion model architecture with attention, transformer, and masking enhancements tailored for tabular data tasks, unifying imputation and generation.

Findings

Outperforms state-of-the-art models like VAE, GAN, and existing diffusion models.

Effectively handles missing data imputation with high efficiency.

Generates statistically similar data with reduced privacy risks.

Abstract

Data imputation and data generation have important applications for many domains, like healthcare and finance, where incomplete or missing data can hinder accurate analysis and decision-making. Diffusion models have emerged as powerful generative models capable of capturing complex data distributions across various data modalities such as image, audio, and time series data. Recently, they have been also adapted to generate tabular data. In this paper, we propose a diffusion model for tabular data that introduces three key enhancements: (1) a conditioning attention mechanism, (2) an encoder-decoder transformer as the denoising network, and (3) dynamic masking. The conditioning attention mechanism is designed to improve the model's ability to capture the relationship between the condition and synthetic data. The transformer layers help model interactions within the condition (encoder) or synthetic data (decoder), while dynamic masking enables our model to efficiently handle both missing data imputation and synthetic data generation tasks within a unified framework. We conduct a comprehensive evaluation by comparing the performance of diffusion models with transformer conditioning against state-of-the-art techniques, such as Variational Autoencoders, Generative Adversarial Networks and Diffusion Models, on benchmark datasets. Our evaluation focuses on the assessment of the generated samples with respect to three important criteria, namely: (1) Machine Learning efficiency, (2) statistical similarity, and (3) privacy risk mitigation. For the task of data imputation, we consider the efficiency of the generated samples across different levels of missing features.