Transformers Meet Relational Databases

TL;DR

This paper introduces a modular neural message-passing scheme for Transformers that directly models relational databases, enabling end-to-end learning from database systems and outperforming existing models across diverse datasets.

Contribution

The paper presents a novel neural architecture that adheres to the relational model, bridging Transformers and relational databases for improved learning from tabular data.

Findings

Superior performance over existing models on multiple datasets

Effective end-to-end learning from database storage systems

Addresses data representation challenges in relational settings

Abstract

Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their extension to the more general case of relational databases. In this paper, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular Transformers from database storage systems. We address the challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results demonstrate a superior performance of this newly proposed class of neural architectures.