REDELEX: A Framework for Relational Deep Learning Exploration

TL;DR

REDELEX is a comprehensive framework for evaluating relational deep learning models on diverse relational databases, providing insights into factors influencing model performance and confirming the superiority of RDL methods.

Contribution

It introduces REDELEX, a new exploration framework that benchmarks various RDL models across over 70 RDBs, analyzing performance factors and making the database collection publicly available.

Findings

RDL models generally outperform classic methods.

Model complexity and database properties significantly affect performance.

The framework provides valuable insights into RDL effectiveness.

Abstract

Relational databases (RDBs) are widely regarded as the gold standard for storing structured information. Consequently, predictive tasks leveraging this data format hold significant application promise. Recently, Relational Deep Learning (RDL) has emerged as a novel paradigm wherein RDBs are conceptualized as graph structures, enabling the application of various graph neural architectures to effectively address these tasks. However, given its novelty, there is a lack of analysis into the relationships between the performance of various RDL models and the characteristics of the underlying RDBs. In this study, we present REDELEX$-$a comprehensive exploration framework for evaluating RDL models of varying complexity on the most diverse collection of over 70 RDBs, which we make available to the community. Benchmarked alongside key representatives of classic methods, we confirm the generally superior performance of RDL while providing insights into the main factors shaping performance, including model complexity, database sizes and their structural properties.