# Chunked TabPFN: Exact Training-Free In-Context Learning for Long-Context Tabular Data

## TL;DR

This paper introduces Chunked TabPFN, a novel approach that enables TabPFN to process long tabular data contexts efficiently without pre-processing, outperforming traditional models on benchmarks.

## Contribution

The paper proposes a tiled-block attention strategy for TabPFN, allowing exact training-free in-context learning on long tabular data without context compression.

## Key findings

- Effective long-context processing on TabArena benchmark
- Outperforms traditional tree-based models on several benchmarks
- Compatible with standard GPU setups

## Abstract

TabPFN v2 achieves better results than tree-based models on several tabular benchmarks, which is notable since tree-based models are usually the strongest choice for tabular data. However, it cannot handle more than 10K context tokens because transformers have quadratic computation and memory costs. Unlike existing approaches that rely on context compression, such as selecting representative samples via K-nearest neighbors (KNN), we introduce a tiled-block strategy to compute attention within the TabPFN framework. This design is compatible with standard GPU setups and, to the best of our knowledge, is the first to enable TabPFN to process long contexts without any pre-processing. We demonstrate the effectiveness of our approach on the standard TabArena benchmark, with code available at https://github.com/mrsergazinov/chunk_tabpfn.