RelBench v2: A Large-Scale Benchmark and Repository for Relational Data

TL;DR

RelBench v2 introduces an extensive, scalable benchmark with new datasets and tasks for relational deep learning, enabling systematic evaluation of models on large, complex relational data.

Contribution

It expands the RelBench benchmark with large-scale datasets, new autocomplete tasks, and integration of external evaluation frameworks for comprehensive RDL assessment.

Findings

RDL models outperform single-table baselines on various tasks.

New datasets increase benchmark size to over 22 million rows.

Autocomplete tasks challenge models to infer missing relational data.

Abstract

Relational deep learning (RDL) has emerged as a powerful paradigm for learning directly on relational databases by modeling entities and their relationships across multiple interconnected tables. As this paradigm evolves toward larger models and relational foundation models, scalable and realistic benchmarks are essential for enabling systematic evaluation and progress. In this paper, we introduce RelBench v2, a major expansion of the RelBench benchmark for RDL. RelBench v2 adds four large-scale relational datasets spanning scholarly publications, enterprise resource planning, consumer platforms, and clinical records, increasing the benchmark to 11 datasets comprising over 22 million rows across 29 tables. We further introduce autocomplete tasks, a new class of predictive objectives that require models to infer missing attribute values directly within relational tables while respecting temporal constraints, expanding beyond traditional forecasting tasks constructed via SQL queries. In addition, RelBench v2 expands beyond its native datasets by integrating external benchmarks and evaluation frameworks: we translate event streams from the Temporal Graph Benchmark into relational schemas for unified relational-temporal evaluation, interface with ReDeLEx to provide uniform access to 70+ real-world databases suitable for pretraining, and incorporate 4DBInfer datasets and tasks to broaden multi-table prediction coverage. Experimental results demonstrate that RDL models consistently outperform single-table baselines across autocomplete, forecasting, and recommendation tasks, highlighting the importance of modeling relational structure explicitly.