CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation

TL;DR

CACTI is a novel masked autoencoding method that improves tabular data imputation by utilizing missingness patterns and feature semantics, outperforming existing techniques across various datasets.

Contribution

The paper introduces CACTI, a new approach that combines copy masking and contextual feature information to enhance imputation accuracy in tabular data.

Findings

CACTI achieves an average 7.8% R^2 improvement over state-of-the-art methods.

It performs well under different missingness conditions, including MNAR, MAR, and MCAR.

Leveraging dataset-specific contextual information significantly boosts imputation performance.

Abstract

We present CACTI, a masked autoencoding approach for imputing tabular data that leverages the structure in missingness patterns and contextual information. Our approach employs a novel median truncated copy masking training strategy that encourages the model to learn from empirical patterns of missingness while incorporating semantic relationships between features - captured by column names and text descriptions - to better represent feature dependence. These dual sources of inductive bias enable CACTI to outperform state-of-the-art methods - an average $R^2$ gain of 7.8% over the next best method (13.4%, 6.1%, and 5.3% under missing not at random, at random and completely at random, respectively) - across a diverse range of datasets and missingness conditions. Our results highlight the value of leveraging dataset-specific contextual information and missingness patterns to enhance imputation performance.