FoMo-0D: A Foundation Model for Zero-shot Tabular Outlier Detection

TL;DR

FoMo-0D is a pre-trained foundation model that enables zero-shot outlier detection on tabular data, eliminating the need for model selection, hyperparameter tuning, or additional training, and demonstrating high accuracy and efficiency.

Contribution

We introduce FoMo-0D, a novel foundation model pre-trained on synthetic data that performs zero-shot outlier detection on tabular data without fine-tuning.

Findings

Outperforms most baselines on 57 datasets

Requires only 7.7 ms per sample for inference

Achieves at least 7x speed-up over previous methods

Abstract

Outlier detection (OD) has a vast literature as it finds numerous real-world applications. Being an unsupervised task, model selection is a key bottleneck for OD without label supervision. Despite a long list of available OD algorithms with tunable hyperparameters, the lack of systematic approaches for unsupervised algorithm and hyperparameter selection limits their effective use in practice. In this paper, we present FoMo-0D, a pre-trained Foundation Model for zero/0-shot OD on tabular data, which bypasses the hurdle of model selection altogether. Having been pre-trained on synthetic data, FoMo-0D can directly predict the (outlier/inlier) label of test samples without parameter fine-tuning -- requiring no labeled data, and no additional training or hyperparameter tuning when given a new task. Extensive experiments on 57 real-world datasets against 26 baselines show that FoMo-0D is highly competitive; outperforming the majority of the baselines with no statistically significant difference from the 2nd best method. Further, FoMo-0D is efficient in inference time requiring only 7.7 ms per sample on average, with at least 7x speed-up compared to previous methods. To facilitate future research, our implementations for data synthesis and pre-training as well as model checkpoints are openly available at https://github.com/A-Chicharito-S/FoMo-0D.