Representation Learning on Out of Distribution in Tabular Data

TL;DR

This paper introduces TCL, a lightweight contrastive learning method for tabular data that effectively handles out-of-distribution data on standard CPU hardware, outperforming existing models in classification tasks.

Contribution

The paper presents TCL, a novel contrastive learning approach tailored for tabular data that is efficient, hardware-friendly, and improves OOD detection and generalization.

Findings

TCL outperforms FT-Transformer and ResNet in classification accuracy.

TCL maintains competitive regression performance.

TCL requires significantly less computational resources.

Abstract

The open-world assumption in model development suggests that a model might lack sufficient information to adequately handle data that is entirely distinct or out of distribution (OOD). While deep learning methods have shown promising results in handling OOD data through generalization techniques, they often require specialized hardware that may not be accessible to all users. We present TCL, a lightweight yet effective solution that operates efficiently on standard CPU hardware. Our approach adapts contrastive learning principles specifically for tabular data structures, incorporating full matrix augmentation and simplified loss calculation. Through comprehensive experiments across 10 diverse datasets, we demonstrate that TCL outperforms existing models, including FT-Transformer and ResNet, particularly in classification tasks, while maintaining competitive performance in regression problems. TCL achieves these results with significantly reduced computational requirements, making it accessible to users with limited hardware capabilities. This study also provides practical guidance for detecting and evaluating OOD data through straightforward experiments and visualizations. Our findings show that TCL offers a promising balance between performance and efficiency in handling OOD prediction tasks, which is particularly beneficial for general machine learning practitioners working with computational constraints.