JoLT: Joint Probabilistic Predictions on Tabular Data Using LLMs

TL;DR

JoLT leverages Large Language Models to perform joint probabilistic predictions on tabular data, effectively handling heterogeneous data types, missing data, and multiple targets without additional training.

Contribution

The paper introduces JoLT, a novel method that uses LLMs for joint probabilistic predictions on tabular data, eliminating the need for data preprocessing or model training.

Findings

JoLT outperforms existing methods on classification and regression tasks.

JoLT can automatically handle missing data and perform data imputation.

The approach is simple, general, and effective for various real-world prediction problems.

Abstract

We introduce a simple method for probabilistic predictions on tabular data based on Large Language Models (LLMs) called JoLT (Joint LLM Process for Tabular data). JoLT uses the in-context learning capabilities of LLMs to define joint distributions over tabular data conditioned on user-specified side information about the problem, exploiting the vast repository of latent problem-relevant knowledge encoded in LLMs. JoLT defines joint distributions for multiple target variables with potentially heterogeneous data types without any data conversion, data preprocessing, special handling of missing data, or model training, making it accessible and efficient for practitioners. Our experiments show that JoLT outperforms competitive methods on low-shot single-target and multi-target tabular classification and regression tasks. Furthermore, we show that JoLT can automatically handle missing data and perform data imputation by leveraging textual side information. We argue that due to its simplicity and generality, JoLT is an effective approach for a wide variety of real prediction problems.