Contrastive Mixup: Self- and Semi-Supervised learning for Tabular Domain

TL;DR

This paper introduces Contrastive Mixup, a semi-supervised learning framework tailored for tabular data, which enhances classification performance with limited labeled data by combining Mixup augmentation and contrastive learning.

Contribution

It proposes a novel semi-supervised approach for tabular data that integrates Mixup-based augmentation with contrastive learning and label propagation, addressing domain-specific challenges.

Findings

Effective on public tabular datasets

Improves performance with limited labeled data

Utilizes label propagation for better pair selection

Abstract

Recent literature in self-supervised has demonstrated significant progress in closing the gap between supervised and unsupervised methods in the image and text domains. These methods rely on domain-specific augmentations that are not directly amenable to the tabular domain. Instead, we introduce Contrastive Mixup, a semi-supervised learning framework for tabular data and demonstrate its effectiveness in limited annotated data settings. Our proposed method leverages Mixup-based augmentation under the manifold assumption by mapping samples to a low dimensional latent space and encourage interpolated samples to have high a similarity within the same labeled class. Unlabeled samples are additionally employed via a transductive label propagation method to further enrich the set of similar and dissimilar pairs that can be used in the contrastive loss term. We demonstrate the effectiveness of the proposed framework on public tabular datasets and real-world clinical datasets.