RDB2G-Bench: A Comprehensive Benchmark for Automatic Graph Modeling of Relational Databases

TL;DR

This paper introduces RDB2G-Bench, a comprehensive benchmark framework for evaluating automatic graph modeling methods of relational databases, highlighting performance variations and structural patterns affecting model effectiveness.

Contribution

It presents the first benchmark for RDB-to-graph modeling, providing extensive datasets, evaluation of multiple methods, and insights into structural factors influencing performance.

Findings

Best graph model outperforms heuristic by up to 10%.

Benchmarking is 380x faster with precomputed datasets.

Structural patterns significantly impact model effectiveness.

Abstract

Recent advances have demonstrated the effectiveness of graph-based learning on relational databases (RDBs) for predictive tasks. Such approaches require transforming RDBs into graphs, a process we refer to as RDB-to-graph modeling, where rows of tables are represented as nodes and foreign-key relationships as edges. Yet, effective modeling of RDBs into graphs remains challenging. Specifically, there exist numerous ways to model RDBs into graphs, and performance on predictive tasks varies significantly depending on the chosen graph model of RDBs. In our analysis, we find that the best-performing graph model can yield up to a 10% higher performance compared to the common heuristic rule for graph modeling, which remains non-trivial to identify. To foster research on intelligent RDB-to-graph modeling, we introduce RDB2G-Bench, the first benchmark framework for evaluating such methods. We construct extensive datasets covering 5 real-world RDBs and 12 predictive tasks, resulting in around 50k graph model-performance pairs for efficient and reproducible evaluations. Thanks to our precomputed datasets, we were able to benchmark 10 automatic RDB-to-graph modeling methods on the 12 tasks about 380x faster than on-the-fly evaluation, which requires repeated GNN training. Our analysis of the datasets and benchmark results reveals key structural patterns affecting graph model effectiveness, along with practical implications for effective graph modeling. Our datasets and code are available at https://github.com/chlehdwon/RDB2G-Bench.