iLTM: Integrated Large Tabular Model

TL;DR

iLTM is a unified large model for tabular data that combines tree embeddings, neural networks, and retrieval, achieving superior performance across diverse datasets with less tuning.

Contribution

The paper introduces iLTM, a novel integrated architecture that unifies multiple methods for tabular data, trained on extensive datasets, and outperforms existing models.

Findings

iLTM outperforms GBDTs and deep models on classification tasks.

Light fine-tuning enables effective transfer to regression tasks.

The model requires less task-specific tuning than competitors.

Abstract

Tabular data underpins decisions across science, industry, and public services. Despite rapid progress, advances in deep learning have not fully carried over to the tabular domain, where gradient-boosted decision trees (GBDTs) remain a default choice in practice. We present iLTM, an integrated Large Tabular Model that unifies tree-derived embeddings, dimensionality-agnostic representations, a meta-trained hypernetwork, multilayer perceptrons (MLPs), and retrieval within a single architecture. Pretrained on more than 1,800 heterogeneous classification datasets, iLTM achieves consistently superior performance across tabular classification and regression tasks, from small datasets to large and high-dimensional tasks. After light fine-tuning, the meta-trained hypernetwork transfers to regression targets, matching or surpassing strong baselines. Extensive experiments show that iLTM outperforms well-tuned GBDTs and leading deep tabular models while requiring less task-specific tuning. By bridging the gap between tree-based and neural methods, iLTM offers a new framework for tabular foundation models for robust, adaptable, and scalable tabular learning.