Graph-Based Feature Augmentation for Predictive Tasks on Relational Datasets

TL;DR

This paper introduces ReCoGNN, an automated framework that enhances relational datasets with features extracted via graph neural networks, improving predictive performance on classification and regression tasks.

Contribution

ReCoGNN is a novel end-to-end automated feature augmentation method that models intra-table dependencies and inter-row relationships using graph neural networks.

Findings

ReCoGNN outperforms existing methods on multiple datasets.

It improves predictive accuracy for both classification and regression.

The framework effectively captures semantic dependencies within tables.

Abstract

Data has become a foundational asset driving innovation across domains such as finance, healthcare, and e-commerce. In these areas, predictive modeling over relational tables is commonly employed, with increasing emphasis on reducing manual effort through automated machine learning (AutoML) techniques. This raises an interesting question: can feature augmentation itself be automated and identify and utilize task-related relational signals? To address this challenge, we propose an end-to-end automated feature augmentation framework, ReCoGNN, which enhances initial datasets using features extracted from multiple relational tables to support predictive tasks. ReCoGNN first captures semantic dependencies within each table by modeling intra-table attribute relationships, enabling it to partition tables into structured, semantically coherent segments. It then constructs a heterogeneous weighted graph that represents inter-row relationships across all segments. Finally, ReCoGNN leverages message-passing graph neural networks to propagate information through the graph, guiding feature selection and augmenting the original dataset. Extensive experiments conducted on ten real-life and synthetic datasets demonstrate that ReCoGNN consistently outperforms existing methods on both classification and regression tasks.

Tables

Table 1. Table 1. Statistics of Datasets

Dataset	Tables	Features	Rows	Task
Olist	4	25	401K	Class.
MovieLens	3	29	1M	Class.
Loyal	4	58	118K	Class.
PED	3	50	18K	Class.
Event	5	129	46K	Class.
Event-Not	5	129	46K	Class.
F1	9	62	133K	Reg.
IMDB	7	45	203K	Reg.
Restbase	3	9	28K	Reg.
Bio	5	11	21K	Reg.

Table 2. Table 2. Results of the Classification Task (bold: the best; underline: the second best; Acc is Accuracy ; AUC is AUC-ROC )

Method	Olist		MovieLens		Loyal		PED		Event		Event-Not
	Acc	AUC-ROC	Acc	AUC-ROC	Acc	AUC-ROC	Acc	AUC-ROC	Acc	AUC-ROC	Acc	AUC-ROC
Base	0.852	0.8296	0.7036	0.5213	0.5424	0.5111	0.515	0.5257	0.7323	0.5022	0.948	0.5443
All	0.8339	0.5616	0.7517	0.5001	0.5353	0.5146	0.6069	0.5842	0.7456	0.5701	0.9757	0.5344
Random	0.8335	0.5113	0.7216	0.5547	0.5203	0.5034	0.5344	0.5437	0.7396	0.6526	0.9736	0.5500
MI	0.8335	0.5506	0.7216	0.5281	0.5203	0.5088	0.6069	0.5326	0.7505	0.5001	0.9729	0.5165
BE	0.8527	0.5001	0.7525	0.5001	0.5421	0.5001	0.5336	0.5007	-	-	-	-
XGBoost	0.9068	0.8852	0.7011	0.5988	0.5421	0.5575	0.6625	0.5337	0.7352	0.6305	0.9729	0.7691
LightGBM	0.9130	0.8984	0.7051	0.6245	0.5421	0.5339	0.6218	0.5834	0.7352	0.6068	0.9729	0.6966
RF	0.8516	0.8677	0.7012	0.6046	0.5355	0.5723	0.6039	0.5041	0.7505	0.6221	0.9727	0.5729
ARDA	0.8526	0.5003	0.7011	0.5048	0.5421	0.5078	0.5276	0.5002	0.7395	0.5023	0.9729	0.5020
ARDA-NoText	0.8526	0.5736	0.7543	0.5066	0.5649	0.5855	0.5461	0.5007	0.7352	0.5173	0.9332	0.5049
LEVA	0.8595	0.5186	0.7152	0.5319	0.52	0.5105	0.5525	0.5528	0.7396	0.5	0.9652	0.5015
ReCoGNN	0.9084	0.8855	0.7980	0.8432	0.5825	0.6397	0.685	0.7462	0.7516	0.6918	0.9603	0.9233

Table 3. Table 3. Results of the Regress Task (bold: the best; underline: the second best)

Method	F1		IMDB		Restbase		Bio
	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE
Base	489.878	248807.529	6.935	50.563	0.445	0.341	0.925	1.316
All	102.814	68337.160	1.539	4.578	0.442	0.345	1.048	1.732
Random	306.206	113842.427	1.546	3.975	0.406	0.323	1.462	3.449
MI	6.734	66.279	1.224	2.406	0.401	0.278	1.80	4.144
BE	15.048	350.338	1.247	2.453	0.411	0.265	3.409	12.875
XGBoost	3.129	13.913	-	-	0.342	0.306	1.241	2.205
LightGBM	3.109	13.679	1.199	2.317	0.336	0.320	1.114	1.177
RF	2.788	14.334	-	-	0.416	0.239	1.213	2.182
ARDA	23.548	1347.496	2.123	5.623	0.407	0.269	3.4	13.457
ARDA-NoText	9.188	103.613	1.246	1.552	0.409	0.268	5.241	31.73
LEVA	4.864	40.4845	1.339	3.017	0.302	0.247	1.28	3.727
ReCoGNN	0.912	3.227	1.113	2.311	0.275	0.230	0.521	0.5826

Table 4. Table 4. Performance of Different Graph Discovery Algorithms on MovieLens Datasets (bold: the best)

Method	$\ell =0.5$		$\ell =0.8$
	Complete	GN	Complete	GN
Average Precision	0.7257	0.7226	0.6988	0.6262
Accuracy	0.7980	0.7781	0.7815	0.7566
F1 Score	0.6738	0.665	0.6118	0.5067
AUC-ROC	0.8432	0.8274	0.8243	0.7689

Equations

$${\mathbf{e}}_i=\{\begin{array}{ll}{f}_{\text{num}}(c_i)\in {\mathbb{R}}^{d_{\text{num}}},& \text{if }\text{type}(c_i)=\text{num}\\ f_{\text{cat}}(c_i)\in {\mathbb{R}}^{d_{\text{cat}}},& \text{if }\text{type}(c_i)=\text{cat}\\ f_{\text{text}}(c_i)\in {\mathbb{R}}^{d_{\text{text}}},& \text{if }\text{type}(c_i)=\text{text}\end{array}$$

$${\mathbf{h}}_j={\mathbf{e}}_j\mathbf{W}\in {\mathbb{R}}^{d_{out}}$$

$$\begin{array}{c}{\mathbf{h}}_u^i=\mathbf{W}{\mathbf{X}}^i[u],\mathbf{W}{\mathbf{h}}_v={\mathbf{X}}^i[v],\\ e_{uv}^i=\text{LeakyReLU}\left({\alpha }^{\top }\left[{\mathbf{h}}_u^i\Vert {\mathbf{h}}_v^i\right]\right),\\ {\alpha }_{uv}^i=\frac{\exp (e_{uv}^i)}{{\sum }_{w\in V}\exp (e_{uw}^i)}\end{array}$$

$${\mathbf{h}}_v^i{}^{\prime }=\sigma (\sum _u{\alpha }_{uv}^i\mathbf{W}{}^{\prime }{\mathbf{h}}_u^i)$$

$${\mathbf{g}}^i=\text{POOL}({\mathbf{h}}_v^i|v\in V)$$

$${\mathcal{L}}_{total}=\sum _{i=1}^B{\mathcal{L}}_i$$

$${\mathbf{A}}_{sum}=\sum _{i=0}^B{\mathbf{A}}_i$$

$$m_i^t=\sum _{j\in N_{v_i}^t}M(h_j,h_{e_{ji}},h_i)$$

$$h_i^{(l+1)}=U(h_i^{(l)},\{m_i^t\mid t\in T_e\})$$

$${\mathcal{L}}_{total}=\sum _{i=1}^N\mathcal{L}(y_i,f_G(x_i))$$

Full text

To address this challenge, we propose an end-to-end automated feature augmentation framework, ReCoGNN, which enhances initial datasets using features extracted from multiple relational tables to support predictive tasks. ReCoGNN first captures semantic dependencies within each table by modeling intra-table attribute relationships, enabling it to partition tables into structured, semantically coherent segments. It then constructs a heterogeneous weighted graph that represents inter-row relationships across all segments. Finally, ReCoGNN leverages message-passing graph neural networks to propagate information through the graph, guiding feature selection and augmenting the original dataset. Extensive experiments conducted on ten real-life and synthetic datasets demonstrate that ReCoGNN consistently outperforms existing methods on both classification and regression tasks.

1. Introduction

Across domains such as finance (Clements et al., 2020), healthcare (Van Walraven et al., 2010), and the internet (Zhang et al., 2019), relational table have emerged as a fundamental format for organizing structured data. To unlock the value embedded in such data, a broad range of predictive modeling tasks have arisen around relational tables. In response, the machine learning community has seen rapid progress in AutoML systems, which aim to automate the process of model selection and hyperparameter tuning. Provided with a dataset and task specification, these systems strive to achieve high performance with limited human involvement.

The effectiveness of the AutoML is greatly influenced by the quality of the user-provided raw data, particularly when the raw data features are insufficient. For instance, an event coordinator seeking to predict attendance for an upcoming event may find that historical data from prior, similar events could be beneficial due to its availability in databases. However, dependence solely on such historical data could result in imprecise predictions. This is attributable to multiple other determinants that could impact attendance numbers, including the event’s geographical setting, the long-term residential trends of expected participants, their enthusiasm for the event’s subject matter, and possibly their social connections, which might sway their decision to attend.

While databases conventionally organize all relevant information into relational tables to facilitate straightforward access, determining which features are predictive in many contexts remains a significant challenge. These predictive features are often dispersed across various interconnected tables within the database. A brute-force approach might involve merging all the potentially relevant tables into a single, extensive table. However, this method is fraught with problems. Primarily, such a large table would encompass an excessive number of features, leading to substantial demands on computational resources and increased processing duration. Ideally, extracting only those features that are genuinely beneficial would be adequate. Nevertheless, selecting attributes from tables to feed into an AutoML system for optimal predictive performance is challenging due to the following reasons:

• •

Feature relevance challenges (Selective Dependency) : In extensive databases, many attributes might lack direct predictive relevance for the target attribute. Including these irrelevant features can introduce noise, masking significant patterns. Beyond irrelevance, some attributes affect the target indirectly through complex correlations. These hidden dependencies, often subtle or combinatorial across various attributes, require in-depth domain knowledge or advanced analytical methods to detect.

• •

Table relationship challenges (Complex Dependency): Effectively leveraging information from multiple related tables introduces another layer of complexity. While the explicitly defined primary key-foreign key relationships offer a primary mechanism for data integration, the task becomes significantly more intricate when relational schema exhibit complex structural patterns. Understanding and navigating these complex inter-table relationships is crucial for building accurate predictive models.

In the face of these challenges, several research efforts have emerged. One major line of work focuses on selecting appropriate features for the predictive task from the merged single table. Classical methods such as Forward Selection and Backward Elimination incrementally add or remove features to optimize model performance. However, they are prone to local optima. Forward Selection may overlook weakly correlated yet useful features, while Backward Elimination may mistakenly remove informative ones in the presence of noise (Chepurko et al., 2020). Methods such as (Kumar et al., 2016; Shah et al., 2017) focus on efficiency by avoiding unnecessary joins, particularly when foreign keys provide most required data. Nevertheless, these strategies are inherently cautious and generally do not enhance performance. ARDA (Chepurko et al., 2020) utilizes data discovery mechanisms to rank potential tables and employs heuristic methods for selecting features. As these rankings do not consider model specifics, their dependability is constrained. Additionally, ARDA is limited to single-hop joins, hindering its ability to leverage intricate multi-hop connections. In contrast, AutoFeature (Liu et al., 2022) and METAM (Galhotra et al., 2023) conceptualize augmentation as a sequential decision-making task, employing reinforcement learning models such as multi-armed bandits and deep Q-networks to dynamically explore join paths, thereby enhancing downstream performance. Nonetheless, these approaches typically entail expensive join operations and repetitive model assessments across diverse tables. Leva (Zhao and Castro Fernandez, 2022), while addressing inter-element relationships in relational databases, presupposes that the importance levels of adjacent elements are constant and not amenable to adaptive learning.

Recent approaches have explored transforming databases into graph structures capable of naturally representing intricate relationships between tables and efficiently managing multihop join paths. The methods proposed by (Cvitkovic, 2020) and (Fey et al., 2023) involve mapping each table tuple to a graph node and employing primary key-foreign key (PK-FK) relationships as graph edges. However, this graph construction technique presumes that all features of the entire table are viewed collectively, neglecting the interaction relationships between individual features.

In this study, we present an innovative end-to-end automated framework named ReCoGNN. ReCoGNN serves as a tool that enhances initial datasets with features extracted from various relational tables, facilitating predictive tasks. The framework operates through three primary phases: Initially, it examines the associations between attributes across all tables (auxiliary tables) that can be joined or indirectly related to the initial data (base table) to discern semantic dependencies. This involves identifying attribute clusters, thereby partitioning the auxiliary tables into segments that are structurally and thematically refined. Secondly, using the segments derived from all auxiliary tables along with the fundamental data, a weighted heterogeneous graph model is constructed. This model characterizes the relational network of records in different table subsets. Lastly, the protocol used is the message passing mechanism of Graph Neural Networks (GNN), which facilitates the dissemination and aggregation of information within the graph structure. This mechanism is used to filter the attributes significant for feature augmentation using learning edge weights, thus enhancing the feature set of the initial data to improve the predictive performance. downstream.

Unlike table-based methods such as (Kumar et al., 2016; Shah et al., 2017; Chepurko et al., 2020), the ReCoGNN framework provides specific advantages. (I) The utilization of a graph structure enables the depiction of diverse entities and their interactions via nodes and edges, resulting in a clear and concise representation of one-to-many, nested, and hierarchical relationships. This method significantly diminishes the complexity associated with join operations found in traditional systems, thus facilitating a more lucid and effective visualization of relationships. (II) The graph-based approach efficiently addresses the problem of data redundancy and inflation typical in conventional techniques. Unlike traditional strategies that require the transformation of all features into a high-dimensional matrix, graph approaches necessitate only the storage of nodes and their interconnections.

In contrast to other graph-based methodologies, the ReCoGNN framework presents distinct advantages by facilitating detailed feature enhancement via an analysis of attribute associations among tables and the identification of attribute groups, which results in more detailed thematic segmentation. It builds a weighted heterogeneous graph model that represents complex relationship networks through advanced weighting and heterogeneity techniques, surpassing basic join operations to improve the robustness and precision of node embeddings. Furthermore, ReCoGNN employs GNN to establish a robust information propagation mechanism, wherein the learning and optimization of edge weights aid in noise filtration and the selection of significant attributes for augmenting features.

Contributions. In summary, the contributions of this paper are:

• •

We propose ReCoGNN, an automated feature enrichment framework that leverages GNNs over relational data graphs to improve prediction tasks with minimal manual effort.

• •

In parsing attribute associations across tables, ReCoGNN identifies attribute groups, splitting the original tables into more refined thematic subsets, allowing the model to more accurately capture semantic dependencies within the data.

• •

We conducted extensive empirical research using multiple real-world and virtual relational datasets to evaluate our proposed method. The experimental results strongly validated the effectiveness and practical value of the ReCoGNN framework.

2. Problem Formulation

**Preliminary. ** Initially, we establish and clarify the essential concepts utilized across this paper.

• •

Base Table : The base table, denoted as $T_{0}=\{a_{1},a_{2},...,a_{n},T\}$, is a unique table that stores the initial data without any augmentation. Each $a_{i}(i\in[1,n])$ represents a feature attribute and $T$ denotes the target attribute.

• •

Auxiliary Tables : The auxiliary tables $\{T_{1},T_{2},\dots,T_{K}\}$ are a set of tables that could join the base table either directly or indirectly through the other auxiliary table. Each auxiliary table $T_{k}={a^{k}_{1},a^{k}_{2},...,a^{k}_{n}}$, where each $a^{k}_{i}(i\in[i,n])$ represents an attribute within the table $T_{k}$.

In a relational table, each tuple represents an entity or relationship instance, with attributes assigned to specific data types (e.g., numerical, string, or categorical). Tables are interconnected through keys: primary keys uniquely identify tuples, and foreign keys reference primary keys to enforce referential integrity.

Predictive tasks on relational tables learn mappings from input attributes to a target attribute. For instance, as shown in Figure  1 the interest attribute in Event-Interest is the target attribute, i.e., the recommendation system wishes to forecast the event of interest for each user. $T_{0}$ is the base table, while $T_{1},\dots,T_{4}$ are auxiliary tables. These tasks are categorized as classification (categorical target) or regression (numerical target). For example, predicting the status of a house sale is a classification task, whereas estimating its price is a regression.

Our objective is to predict user interest in specific events. In the Event-Attendees (T4), the invited and yes attributes represent invitation status and confirmed attendance respectively. These indicators reflect event popularity and may influence user interest. Such attributes can correlate with our prediction target, providing valuable auxiliary signals. Identifying attribute correlations across auxiliary tables enhances predictive performance. However, detecting these relationships in complex database structures presents challenges. While some attributes are independently predictive, others gain value only in combination. Feature interactions reveal patterns invisible when examining features in isolation, requiring combinatorial analysis. For example, in Events (T3), the lat and lng attributes individually offer limited information but jointly determine event location - a factor likely affecting interest. These combinations often embody complex semantic relationships that are difficult to manually define or encode. Such challenges motivate the development of an end-to-end learning framework capable of autonomously discovering synergistic attribute combinations. By enriching base table features through this process, we can improve model performance while minimizing manual feature engineering efforts.

Problem Statement. Given a base table $T_{0}$ with the target attribute and auxiliary tables $\{T_{1},\dots,T_{K}\}$, we aim to augment $T_{0}$ with attribute from $\{T_{k}\}$ to improve prediction accuracy for the target. However, the core challenge lies in identifying relevant attributes from $\{T_{k}\}$ whose integration into $T_{0}$ enhances model performance.

3. Solution Overview

As previously discussed, integrating discriminative features from auxiliary tables into the base table poses two key challenges: assessing feature importance and modeling inter-table relationships. We propose ReCoGNN, a comprehensive framework (illustrated in Figure  2) to address these challenges. In the first phase, ReCoGNN uses a graph neural network to identify task-relevant links among table attributes, segmenting the original tables into semantically aligned sub-tables. In the second phase, ReCoGNN forms a heterogeneous graph incorporating both explicit inter-table connections and implicit similarity ties. A graph neural network is employed to refine the tuple representations in the base table, thereby improving features for the predictive task.

**Stage One. **In the first stage of ReCoGNN, called ”Task-Relevant Table Structuring,” our goal is to extract from each original table a set of sub-tables composed of highly related (and potentially overlapping) attributes. This approach moves away from the traditional practice of treating a table as a monolithic structure, instead generating semantically coherent attribute groups that are more directly aligned with the prediction task. Intuitively, not all tuples are useful for the task, as many cannot be directly or indirectly joined with the base table, rendering them uninformative for prediction. Therefore, we first identify the most task-relevant tuples in each table based on their potential contribution to the prediction objective. By focusing subsequent analysis on this informative subset, we enhance both the efficiency and effectiveness of feature modeling and downstream learning.

**Stage Two. **In the second stage of ReCoGNN, we aim to enhance the feature representation of the base table by integrating information from the sub-tables generated in the first stage. Intuitively, not all sub-tables contribute equally to the prediction task. Thus, the key challenge at this stage is to select the most task-relevant sub-tables and effectively integrate them into the base table. Moreover, since it is difficult to precisely evaluate whether the intra-table relationships identified in the first stage fully align with the prediction task requirements, we assess the effectiveness of feature enhancement based on the final prediction performance in the second stage. Therefore, we design a heterogeneous graph and employ a Graph Neural Network (GNN) to learn edge weights, thereby identifying and filtering attributes that are meaningful for feature enhancement. While achieving this objective, the GNN also aggregates relevant information from associated sub-tables to the base table nodes, ultimately generating highly valuable representations for downstream prediction tasks. The learned representations can be directly applied to different prediction heads (e.g., classifiers or regressors), supporting diverse downstream applications.

4. Task-Relevant Table Structuring

To enhance the predictive performance of the base table, we aim to incorporate useful features from auxiliary tables in the database. Constructing effective attribute sets from auxiliary tables is non-trivial, as individual attributes may not be discriminative alone but become useful in combination, while the tables themselves contain heterogeneous data types.To tackle these obstacles, we introduce a method for modeling intra-table attribute relationships within each supplementary table. Each tuple is viewed as an instance of attribute co-occurrence, enabling the creation of a fully connected graph among attributes. A Graph Attention Network (GAT) is then employed to learn the interaction strengths. Task-relevant attribute combinations are identified through learned edge weights and used to form more focused, informative sub-tables. The methodology involves three steps: (1) pinpointing task-relevant tuples for base table integration to gain supervision; (2) constructing attribute-level graphs for these tuples and training a GAT to capture feature interactions; and (3) deriving highly relevant attribute groups from attention weights to reorganize each supplementary table into a compact, task-specific sub-table.

4.1. Identifying Task-Relevant Tuples

Recall that our goal is to uncover meaningful combinations of attributes within each table that contribute to predicting the target attribute’s value. To achieve this, we analyze the relationships between attribute values within individual tuples of the table. Identifying task-relevant tuples is non-trivial. The natural intuition suggests that a tuple is task-relevant if it can be linked to a target attribute. However, given the complex schema relationships, a single tuple may connect to multiple target values. So the key question to be answered is: Which tuples are task-relevant and useful for learning attribute relationships?

Intuitively, a tuple in an auxiliary table is considered potentially relevant to a target attribute value if it can be linked to a tuple in the base table. This linkage is established through a sequence of join operations defined by the relational schema. However, relational schema can be complex, offering numerous join paths. Many of such join paths may not capture semantically meaningful relationships for our specific prediction task. To address this, we construct meta-path for each auxiliary table to define its semantic relationship to the base table. As described earlier, the join relationships between the base table $T_{0}$ and all auxiliary tables $\{T_{1},...,T_{K}\}$. are defined by the database schema. This relationship can be effectively modeled as a Directed Join Graph (DJG), wherein each node represents a relational table within the dataset $T=\{T_{0}\}\cup\{T_{1},...,T_{K}\}$, while the edges signify executable join operations in $T$, incorporating their respective link types such as $1:1$, $1:n$, and $n:1$. However, an unprocessed DJG presents complex pathways, which complicates the handling of join relationships, particularly those involving circular structures. To effectively identify a meaningful meta-path for each auxiliary table, we employ a greedy path search algorithm starting from the base table $T_{0}$, expanding the path one step at a time. At each step, the algorithm chooses the next join that leads to the highest path score. The path scoring function jointly considers two aspects:

• •

Path length: Shorter paths are generally more interpretable and less likely to introduce redundant or noisy features. The path length score is defined as $S_{L}=\frac{1}{1+L}$, where $L$ is the number of edges in the current path.

• •

Join directionality: One-to-many (1:n) joins are prone to amplifying noise and redundancy, especially when they appear repeatedly in multi-hop paths. To model this, we define a weighted penalty term that accounts for the risk associated with each 1:n join:

$S_{N}=\frac{1}{1+\sum_{i=1}^{L}w_{i}\cdot\mathbb{I}_{\text{1:n}}(e_{i})}$, where $e_{i}$ denotes the $i$-th edge in the path, $\mathbb{I}_{\text{1:n}}(e_{i})$ is an indicator function that equals 1 if the edge is a 1:n join, and $w_{i}$ is a weight reflecting the risk (e.g., average fan-out) of the join.

The overall path score is computed as a weighted sum: $S_{\text{path}}=\alpha S_{L}+\beta S_{N}$ where $\alpha$ and $\beta$ are tunable weights that balance structural simplicity and join stability. Other domain knowledge or schema-level constraints can also be incorporated into this scoring function.

During greedy search, the algorithm expands only the neighbor with the highest incremental path score at each step, avoiding the cost of exhaustive enumeration. Once the auxiliary table is reached, the path with the highest accumulated score is selected as its meta-path.

Given an auxiliary table $T_{i}=\{t_{i1},\dots,t_{in}\}$ and its specified meta-path, if there exists an instance of a join path that successfully links a tuple $t_{ij}$ in $T_{i}$ to a tuple $t_{0k}$ in the base table, we consider tuple $t_{ij}$ relevant to tuple $t_{0k}$ and thus carrying the corresponding target label. Consider the example in Figure 3. As shown in the figure, the base table is Event-Interest, with the target attribute interest and Users serving as an auxiliary table. The tuple with the primary key $user=1$ in the User table obtains the value of the label from the Base table by linking to the tuple in the Event-interest table where the foreign key $user=4$.

Coreset. In real-world scenarios, database tables can contain a vast number of tuples. Utilizing every tuple ensures information completeness but results in extremely high computational costs for further processing. To mitigate this, we employ a sampling strategy: forming a coreset by selecting a representative subset of tuples from each auxiliary table to approximate the entire dataset. This coreset maintains essential structural details while significantly lowering computational costs, albeit with a slight potential compromise in model accuracy. Our approach initiates coreset construction from the base table, selecting core tuples based on criteria such as label distribution. These core tuples are then extended to auxiliary tables along meta-paths.

4.2. Modeling Attribute Relationship with GAT

Building upon the task-relevant tuples identified in Section 4.1, we now turn our attention to discerning the predictive power of individual attributes within these auxiliary tables. However, it is non-trivial task, as the contribution of each attribute to the target prediction task is not equal. Some might be directly indicative of the target, while others might be noisy or only relevant in conjunction with other attributes. To effectively capture these complex inter-attribute dependencies and automatically learn their relevance to the prediction task, we propose to represent each task-relevant tuple as a complete graph.

4.2.1. Construct a Complete Graph for Each Tuple.

For each identified task-relevant tuple in an auxiliary table, we denote its non-key attributes as $\{c_{1},c_{2},\dots,c_{N}\}$. Here we explicitly exclude the primary key of the auxiliary table itself and any foreign keys it might contain, as their inherent values primarily serve for identification and inter-table linkage rather than directly representing features relevant to the prediction task within the scope of this table’s attributes. For such tuple $t$, we construct an individual complete graph $G_{t}=(\mathbf{V}_{t},\mathbf{E}_{t},\mathbf{X}_{t})$. The set of nodes in $V_{t}$ corresponds to the attribute values in $\{c_{1},c_{2},\dots,c_{N}\}$ within the tuple $t$. Given the heterogeneous data types of the attribute values (numerical, categorical, and text), the feature vector $\mathbf{X}_{t}[i]$ for node $v_{i}$ is encoded according to the data type of the corresponding attribute as follows:

Numerical: For all task-relevant tuples, identify the data type by column. For each element $c_{i}$ of the identified numeric type columns (from any tuple $t$), use a model(Gorishniy et al., 2022) with shared parameters to encode it to obtain the encoded result $\mathbf{e}_{i}\in\mathbb{R}^{d_{\text{num}}}$.

Categorical: For each cell $c_{i}$ recognized as belonging to a categorical column, we employ an encoding mechanism utilizing an embedding look-up based encoder to transform it into a vector $\mathbf{e}_{i}\in\mathbb{R}^{d_{\text{cat}}}$.

Text: Regarding text columns, there are several impressive models available, such as Sentence-BERT (Reimers, 2019). Encode the text category $c_{i}$ into a vector $\mathbf{e}_{i}\in\mathbb{R}^{d_{\text{text}}}$.

For each cell $c_{i}$, generate embeddings based on its data type:

[TABLE]

where $f_{\text{num}}$, $f_{\text{cat}}$ and $f_{\text{text}}$ are the numerical encoder, the categorical encoder, and the text encoder with output dimensions $d_{\text{num}}$, $d_{\text{cat}}$, and $d_{\text{text}}$, respectively. In the experiment, we use PyTorch Frame, an advanced modular deep learning extension for PyTorch (Hu et al., 2024).

Embedding dimensions generated by encoders for different cell categories (e.g., numerical, categorical, text) often exhibit significant inconsistencies. Specifically, text-type cells typically yield higher-dimensional embeddings compared to their numerical and categorical counterparts. To address this heterogeneity, deep learning methods such as Multi-Head Self-Attention mechanisms (Vaswani, 2017) or adapted ResNet (Gorishniy et al., 2021) architectures can be employed to project column-wise embeddings into a unified dimensional space, thereby enabling seamless integration for downstream tasks.

Project embeddings of varying dimensions into a unified target dimension $d_{\text{out}}$:

[TABLE]

where $\mathbf{W}$ are learnable projection matrices for dimensionality alignment to $d_{\text{out}}$.

4.2.2. Leveraging GAT for Attribute Relationship

Having constructed a complete graph for each task-relevant tuple, where nodes represent attributes and edges signify potential relationships, the next crucial step is to capture the intricate and task-specific relationships between attributes within each tuple. The general idea is shown in Figure 4. To achieve this, we first build a complete graph for each identified task-relevant tuple. These complete graphs are encoded by a share-parameter GAT. The representation of each graph is fed into a prediction head, aiming to predict the label associated with the corresponding tuple. Unlike methods that treat all nodes uniformly, GATs employ an attention mechanism that allows each node to dynamically learn the importance of its connection with other nodes in the graph in the context of the target prediction task. The learned attention weights on the edges of the complete graphs serve as indicators of the significance of these pairwise node interactions (representing attribute relationships) for predicting the target attribute label.

Given the complete graph $G_{i}(V,E,X_{i})$ constructed from the $i$-th task-relevant tuple, let $u$ and $v$ be two nodes in the graph. The attention weight $\alpha^{i}_{uv}$ is derived as follows:

[TABLE]

where $X^{i}[u]$ and $X^{i}[v]$ are feature vectors for $u$ and $v$. The attention mechanism $\alpha$ and weight matrix $W$ of the GAT layer are shared across all complete graphs. The updated embedding $h_{v}^{i}\,{}^{\prime}$ for node $v$ is computed by aggregating the feature vectors of its neighbors, weighted by the learned attention coefficient

[TABLE]

where $u$ is any other node in the complete graph, $h_{u}$ is the embedding of $u$, $W^{\prime}$ is a learnable weight matrix, and $\sigma$ is a non-linear activation function.

The graph-level representation $\mathbf{g}$ of the complete graph $G_{i}$ is derived by applying a pooling function to the set of its learned node embeddings $\{\mathbf{h}^{i}_{v}\}_{v\in V}$ as follows:

[TABLE]

The resulting graph-level representation $g^{i}$, which encapsulates the learned relationships between attributes within the corresponding task-relevant tuple, is fed into a prediction head. The architecture of this prediction head is specifically tailored to the type of the target attribute in the base table. The specific architecture of the prediction head is orthogonal to our approach. Any standard network suitable for the target task (classification or regression) can be employed. For a regression task, the prediction head outputs a continuous value $\hat{y_{i}}$, and the loss $\mathcal{L}_{i}$ is computed using Mean Squared Error (MSE) between $\hat{y_{i}}$ and the true target attribute value $y_{i}$. In contrast, if it is a classification task, the prediction head outputs a probability distribution over the possible classes $\hat{p}_{i}$, and the loss $\mathcal{L}_{i}$ is computed using cross entropy loss between $\hat{p}_{i}$ and the one-hot true class label $y_{i}$. The total loss is computed by aggregating the individual prediction losses as follows:

[TABLE]

The GAT mechansim learns attention weights for every edge in each complete graph. These learned weights are derived directly during the optimization process aimed at minimizing the prediction loss. Thus, they provide a valuable insight into the contribution of each attribute pair to the final prediction. Higher weights signify more influential attribute relationships for the predictive task. These learned attribute relationships will be leveraged to group attributes into more focused sub-tables.

4.3. Group Attributes into Sub-tables

Having modeled the pairwise relationships between attributes within each task-relevant tuple using GAT and obtained edge weights reflecting their predictive significance, we now aim to leverage this information to structure the original auxiliary tables into more focused, task-relevant sub-tables. To begin with, we first introduce how to extract significant attribute relationships based on the edge weights learned from all task-relevant tuples.

4.3.1. Extracting Significant Attribute Relations.

For each task-relevant tuple $t_{i}$, we use $\mathbf{A}_{i}$ to denote the learned edge weights, where each entry $\mathbf{A}_{i}[u][v]=\alpha^{i}_{u,v}$, the attention weight between $u$ and $v$ in graph $g^{i}$. To consolidate the importance of these pairwise relationships across all task-relevant tuples, we compute a cumulative attribute relationship matrix $\mathbf{A}_{sum}$ by summing the individual weight matrices:

[TABLE]

To ensure a consistent scale for identifying significant relationships, we then normalize the values in $\mathbf{A}_{sum}$ to the range $[0,1]$ using min-max normalization. This normalized matrix $\mathbf{A}_{norm}$ reflects the overall importance of each pairwise attribute relationship across the entire set of task-relevant tuples. We then identify the most important attribute relationships by applying a predefined threshold $\ell$. An edge between node $u$ and $v$ is considered significant if its corresponding weight $\mathbf{A}_{norm}[u][v]$ is greater than $\ell$. We use $E_{sig}$ to denote the set of selected edges. These selected edges represent the attribute pairs that the model has consistently deemed most relevant for predicting the target variable across all considered contexts.

Identifying significant pairwise attribute relationships provides valuable insights, yet directly utilizing the original wide table for downstream tasks can still be suboptimal. A key challenge lies in the fact that not all identified relationships might be equally relevant in all contexts, and the presence of weakly related or irrelevant attributes within the same table can introduce noise and dilute the more potent signals. Therefore, organizing the attributes into more focused sub-tables, based on their learned relationships, holds the potential to create more semantically coherent and task-relevant data segments.

4.3.2. Extracting sub-tables

Motivated by the potential benefits of focusing on strongly related attributes, we now introduce our strategy for extracting more targeted sub-tables from the original auxiliary table. To achieve this extraction, we construct a graph $G_{sig}(\mathbf{V},\mathbf{E}_{sig})$ to explicitly model the relationship between attributes based on the set $\mathbf{E}_{sig}$ of selected significant edges. By analyzing the structure of this graph formed by gathering all significant edges, we aim to discover tightly knit groups of highly relevant attributes. Specifically, we propose to identify the set of maximal cliques $\{C_{1},C_{2},\dots,C_{t}\}$ within $G_{sig}$. Let be the set of identified maximal cliques. For each maximal clique $C_{i}$, the attributes corresponding to the nodes within it form a sub-table. The rationale behind using maximal cliques is that each clique represents a subset of attributes where every attribute is significantly related to every other attribute within that subset, thus forming a highly cohesive and potentially task-relevant sub-table.

Example 0.

As shown in Figure 6, in the overlap scenario, the attributes of the auxiliary table $T_{1}$ form the graph $G_{sig}$. In this graph, the nodes $C_{1}$, $C_{2}$, and $C_{3}$ are all interconnected, forming a maximal clique $\{C_{1},C_{2},C_{3}\}$, which constitutes a sub-table $T_{1,2}$.

Remark. It is important to note that this strategy may result in sub-tables that are either disjoint or overlapping. As shown in Figure 5, Disjoint produces two disjoint sub-tables, while Overlap produces two overlapping sub-tables. This potential for disjointness and overlap reflects the complex and potentially multifaceted nature of attribute relationships relevant to the prediction task.

5. Graph-based Data Augmentation

Following the modeling of intra-table attribute relationships to capture semantic dependencies and partition tables into more focused segments, as detailed in the previous section, we now turn our attention to the crucial task of modeling inter-table relationships. In this section, we introduce a approach to construct a heterogeneous graph that explicitly encodes the connections between tuples across these partitioned tables and the base table. This enables us to select task-relevant attributes and augment init tuples that span the entire relational schema.

5.1. Constructing the Inter-Table Graph

While the partitioned sub-tables offer more focused representations of information within individual tables, the true predictive power often lies in the intricate relationships between entities residing across these different tables. To effectively capture such inter-table dependencies, a graph-based learning emerges as a natural and intuitive choice. However, the construction of such a graph in our context presents two key challenges that necessitate careful consideration:

• •

Original v.s. Splitted Tuples. How do we effectively link base table tuples to relevant information distributed across the partitioned auxiliary tables? Moreover, how are inter-table relationships accurately modeled when a single original tuple is represented by multiple sub-tuples, ensuring proper information aggregation?

• •

**Supervision Isolation.**Since only base table nodes are labeled and the graph structure is defined exclusively by join relationships, semantically similar nodes are often disconnected in the topology, which limits the propagation of supervision signals and hinders the learning of consistent representations.

To address these two key challenges, we propose a novel approach for constructing a heterogeneous graph. The heterogeneous graph, denoted by $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{X})$, is constructed as follows:

Nodes. Each tuple in the base table $T_{0}$ and each tuple within the partitioned sub-tables $T_{i\_1},\dots,T_{i\_j}$ of the auxiliary tables $T_{i}$ constitutes a node in $\mathcal{G}$. Thus, the set of nodes $\mathcal{V}$ is the union of all tuples from the base tables and all sub-tables from the partitioned auxiliary tables.

Features. Each node in the heterogeneous graph, representing a tuple, is associated with a feature vector derived from its constituent attribute values. Given the diverse nature of attributes in relational tables, which can be numerical, categorical, or textual, we employ modality-specific encoders to generate meaningful embeddings for each attribute. Following the idea in (Fey et al., 2023), we use a modality-specific encoder to embed each attribute into embeddings. Once each attribute within a tuple is embedded, we concatenate these attribute embeddings to form the final feature vector for the node.

Edges. We define two primary types of edges within our heterogeneous graph, capturing both implicit relationships within the base table and explicit, schema-defined connections across tables:

• •

Explicit Edges(Inter-Table Connections): Consider tuple $t_{u}$ from one table and tuple $t_{v}$ from a different table. We add an edge between $t_{u}$ and $t_{v}$ if tuple $t_{u}$ and $t_{v}$ can be joined based on the primary and foreign key relationship defined in the database schema. Moreover, if an original auxiliary table $T_{i}$ is partitioned into sub-tables $T_{i1},\dots,T_{im}$, a tuple $t$ from $T_{i}$ is correspondingly represented by sub-tuples $t_{1}\in T_{i1}$, $t_{2}\in T_{i2}$, $\dots$, $t_{m}\in T_{im}$. If a tuple $t^{\prime}$ from another table can be joined with the original tuple $t$, we create edges from $t^{\prime}$ to each of the sub-tuples of $t$, i.e., $(t^{\prime},t_{1}),\dots,(t^{\prime},t_{m})$. This ensures that information from a joinable tuple is propagated to all the tuples from the partitioned sub-tables.

• •

Implicit Edges(within the Base Table): Tuples in the base table might represent entities that are semantically similar or related in ways not directly encoded by joins. For instance, two customer records might have very similar purchasing histories or demographic profiles, even if there isn’t a direct foreign key relationship linking them. To capture such semantic similarity, we create edges between tuples in the base table based on the similarity between their corresponding feature vectors. Specifically, for any two tuples from the base table $T_{0}$, we create an edge between them if the similarity between their features is above a predefined threshold $\theta$. Alternatively, we can adopt a more selective approach by connecting each tuple to its top-$K$ most similar tuples within the base table. By limiting the number of implicit connections per node, we can create a sparser graph compared to a threshold-based approach. This can lead to more efficient message passing during GNN training and potentially reduce computational overhead.

5.2. Augmenting Table

The constructed heterogeneous graph serves as a comprehensive representation of the intricate relationships among tuples spanning the base table and all partitioned auxiliary tables. However, to effectively leverage this complex structure for both feature selection and feature enhancement, we require a mechanism capable of performing information propagation and aggregation over the graph, thereby learning fine-grained edge weights that capture these diverse relational patterns.

GNNs perform message passing to allow each node to gather information from its neighbors, weighted by the edge connections, and iteratively refine its own representation. This process enables the model to learn high-order relationships and dependencies that are implicitly encoded within the graph structure, ultimately yielding tuple embeddings that are informed by the broader relational context and tailored for accurate prediction of the target column in the base table.

Consider a heterogeneous graph $G=(\mathbf{V},\mathbf{E},\mathbf{T_{v}},\mathbf{T_{e}})$, where $\mathbf{V}$ is the set of nodes, $\mathbf{E}$ is the set of edges, $\mathbf{T_{v}}$ is the set of node types, and $\mathbf{T_{e}}$ is the set of edge types.

Aggregation Function For a target node $v_{i}$ from the base table and an edge type $t\in T_{e}$, messages are generated from its neighbors $N_{v_{i}}$ based on different edge types:

[TABLE]

Here, $M$ is the message generation function, $h_{j}$ is the feature of neighbor node $j$, $h_{e_{ji}}$ is the edge feature, $h_{i}$ is the feature of node $i$, and $N_{v_{i}}^{t}$ denotes the set of neighbor nodes connected to node $i$ via edges of type $t$.

Message Passing The messages $m_{i}^{t}$ generated from each edge type $t\in T_{e}$ are integrated:

[TABLE]

Here, $U$ is the message updating function, $h_{i}^{(l)}$ denotes the feature of node $i$ at layer $l$, $h_{i}^{(l+1)}$ denotes the feature of node $i$ at layer $l+1$, and $\{m_{i}^{t}\mid t\in T_{e}\}$ is the set of all messages received from different edge types.

Prediction The total loss is computed by aggregating the prediction losses over all labeled nodes:

[TABLE]

where $N$ is the number of labeled nodes, $x_{i}$ denotes the features of node $i$, $y_{i}$ is its ground-truth label, $\mathcal{L}$ is the task-specific loss function, and $f_{G}$ is a heterogeneous graph neural network.

Our research aims to use a GNN to develop advanced representations for tuples in the base table by integrating vital data from auxiliary tables. Given that labels exist solely for the base table nodes, we employ the GNN to spread label data, boosting the base table node representations’ accuracy and predictive ability. Specifically, by refining the graph’s edge weights, we evaluate each edge’s significance to improve predictions, assuming more important edges receive higher weights during training. The central task is to boost the tuple representations’ predictive accuracy from the base table. We test these enriched representations in real-world prediction contexts to pinpoint the most valuable features. This issue is defined as enhancing prediction efficacy according to the formula 9. This strategy results in selecting features linked to high-weight edges, marking them as crucial for the prediction task. The enhanced representations from the base table not only reflect explicit inter-table relationships but also capture hidden semantic connections, thereby achieving superior predictive accuracy amidst intricate patterns across diverse tables.

6. Experimental Study

In this section, we focus on answering the following questions:

RQ1: How does ReCoGNN perform compared to various baseline?

RQ2: How do key parameters affect the performance of ReCoGNN?

RQ3: How do different components contribute to the system’s performance?

6.1. Experimental Settings

Datasets. We use 10 datasets from various domains to evaluate our ReCoGNN. As shown in Table 1, we give the basic information on the datasets. Among the datasets, six of them are for classification tasks and the other four datasets for regression tasks. Now we present the detailed information of these datasets.

• •

Olist (oli, [n.d.]) is derived from the Brazilian e-commerce platform Olist and is utilized for predicting customer satisfaction. A customer is deemed satisfied if their review$\_$score is at least 3.

• •

MovieLens (mov, [n.d.]) is compiled by GroupLens Research from the MovieLens platform aiming to forecast user gender.

• •

Loyal (loy, [n.d.]) aims to enhance credit card holders’ personalized recommendations by employing machine learning for predicting customer loyalty.

• •

PED (ped, [n.d.]) from DonorsChoose.org is an online charity aiding K-12 schools through project-based donations. The task is to identify exceptionally exciting projects.

• •

Event (eve, [n.d.]) includes user actions, event metadata, and other information, in order to predict which events users will be interested in.

• •

Event-Not (eve, [n.d.]) aligns with the event dataset but focuses on predicting the events that are not of user interest.

• •

F1 (f1, [n.d.]) contains comprehensive historical data from all Formula One seasons since 1950. The regression task involves predicting the average position order of a driver.

• •

IMDB (Leis et al., 2015) predicts the scores of movies and the dataset includes individual characteristics of movies.

• •

Restbase (res, [n.d.]) comprises restaurant data, framing the prediction of customer review details as a regression task.

• •

Bio (Blockeel et al., 2004) dataset is utilized for a regression task aimed at estimating ”the molecule’s bioactivity”.

Baseline. Our methodology is evaluated against various solution.

1. (1)

Base retains only the native features of the base table. 2. (2)

All simply joins all auxiliary tables with the base table. 3. (3)

Random randomly selects $K$ features from auxiliary tables. 4. (4)

Mutual Information (MI) is the filter-based feature selection method that evaluates the importance of features through different mechanisms and selects the top-$K$ feature subset. 5. (5)

Backward Elimination (BE) (Guyon and Elisseeff, 2003) is a wrapper-based feature augmentation method that starts by joining all auxiliary tables and then iteratively removes the features that most degrade model performance. The final subset is achieved by step-by-step exclusion of the least useful features. 6. (6)

XGBoost, LightGBM and Random Forest (RF) (Chen and Guestrin, 2016; Ke et al., 2017; Breiman, 2001) are embedded feature selection methods that combine the advantages of filter methods and wrapper methods by automatically performing feature selection during the model training process. 7. (7)

ARDA (Chepurko et al., 2020) combines the methods of building the core set and selecting random injection features, effectively screening out features that contribute to improving model accuracy while avoiding the introduction of noisy features. As the ARDA-NoText algorithm is incapable of directly handling text-type data, we implement two approaches: either converting text attributes to high-dimensional vectors (ARDA) or eliminating them entirely (ARDA-NoText). 8. (8)

LEVA (Zhao and Castro Fernandez, 2022) is an end-to-end system that constructs relational embeddings to represent relational data as vectors, which are then used to efficiently describe the base table. It constructs a graph to represent relational data and embeds this graph into high-dimensional space.

Evaluation Metrics. Model performance evaluation is conducted using the following methodology: For classification tasks, primary metrics include AUC-ROC and Accuracy, complemented by F1 Score and Average Precision for a detailed performance analysis in parametric sensitivity studies. In regression tasks, evaluation relies on Mean Squared Error (MAE) and Mean Absolute Error (MSE).

In this research, we applied random sampling to the initial datasets, namely Olist, Event (Event-Not), PED, and Loyal, because of their large size.

Implementation. In our experiment, we chose Python as the programming language. For the GNN part, we used the PyTorch-Geometric library, specifically adopting the GraphSAGE(Hamilton et al., 2017) and GAT(Veličković et al., 2017) models. To encode the relational tables, we utilized the PyTorch Frame (Hu et al., 2024) library. Furthermore, training of the XGBoost and LightGBM baseline models also directly used the interfaces provided by the PyTorch Frame library. Unless otherwise specified, the default threshold $\ell$ for extracting important attribute relationships is set to 0.8. Each experimental trial was set with an 18-hour time limit, at which point the process would automatically conclude without achieving the results.

6.2. Comparison with Baselines

Here, we evaluate the performance of the ReCoGNN model against other baselines, focusing on classification and regression tasks.

**Classification. ** As illustrated in Table 2, we assess a variety of baseline methods across multiple datasets and utilizing different models. We use Accuracy and AUC-ROC metrics to evaluate each dataset across different models. Accuracy provides a straightforward and easily understandable performance metric, while AUC-ROC complements it by being effective in situations with imbalanced data. Table 2 shows the effectiveness of different methods on the six datasets. the performance of the base methods, including Base Table, Big Table, and Random, was inferior to that of embedding-based methods (including XGBoost, LightGBM, and RF) as well as the ReCoGNN method. The experimental results indicate that using only the Base Table generally yielded poor results, and in the Olist dataset, the All method performed even worse than the Base Table, with the AUC-ROC significantly dropping from 0.8296 to 0.5616. This is because, although the All method added more features through join operations, the unfiltered features introduced noise. Meanwhile, the all-table join operation might also lead to data drift issues. The performance of the Random method was unstable: in the Loyal dataset, its metrics were lower than those of the Base and All, while in the Event-Not dataset, it performed better than the Base. This indicates that randomly selecting features might yield useful features or might introduce noise, thus systematic feature selection is necessary.

Embedding-based methods (XGBoost, LightGBM, and RF) performed the best among all baselines. For example, in the Olist dataset, LightGBM achieved the best results in both Accuracy and AUC-ROC metrics. The advantage of these methods lies in their ability to dynamically evaluate feature importance during training and gradually optimize the feature subset. However, the downside is the high computational cost: XGBoost (100 epochs) took 35,899.234 seconds, LightGBM (100 epochs) took 225,340.247 seconds, and ReCoGNN (200 epochs) took 9,284.666 seconds.

The mutual information method based on filtering performed worse than the embedding methods in all experiments. For instance, in the Loyal dataset, the AUC-ROC of the mutual information method was only 0.5088, significantly lower than that of XGBoost (0.5575), LightGBM (0.5339), and RF (0.5723). This performance gap stems from two main factors: firstly, filtering methods evaluate features solely based on univariate statistics, completely ignoring the interactions among features; secondly, their feature selection process is independent of model training and does not obtain real-time feedback on prediction performance. It is particularly noteworthy that traditional filtering methods lack effective mechanisms to handle text-type features (which need to be encoded into $300$-dimensional vectors), further limiting their performance.

The BE method based on wrappers, although iteratively referencing model feedback to select features, neither outperformed the mutual information method nor reached the level of embedding methods. This method has two main deficiencies: first, constrained by a greedy search strategy, it has difficulty fully exploring the feature space and is prone to local optima; second, similar to filtering methods, it faces technical bottlenecks when handling high-dimensional text features. These limitations prevent the BE method from discovering the optimal feature combination, ultimately affecting overall prediction performance.

The performance of ARDA and ARDA-NoText was inferior to embedding-based methods and our method in all datasets. This is attributable to three main factors: first, the original ARDA-NoText algorithm does not support text feature processing, and the modified ARDA-NoText version in the baseline experiments sacrificed some performance for this; second, ARDA-NoText avoided processing issues by removing text features but led to the loss of effective feature information; finally, although the Leva method achieved better Accuracy values than other baselines in most datasets due to its graph structure design, its performance improvement was limited by the inability to fully explore potential relationships among features.

Our proposed ReCoGNN solution outperformed all baseline methods in terms of Accuracy and AUC-ROC metrics in the Movie, Loyal, and PED datasets, and also demonstrated competitive advantages in other datasets. The technical advantages of ReCoGNN lie in two aspects: 1) deeply exploring the potential relationships among features within the same table and finely splitting auxiliary tables; 2) accurately modeling join relationships among tables through a weighted heterogeneous graph structure, thus achieving more precise feature enrichment. These mechanisms collectively ensure the outstanding performance of the ReCoGNN method.

**Regression. ** Using the same baselines as the classification experiments to construct regression tasks, Table 3 presents the experimental results, with performance evaluation using MAE and MSE. The trend of results is consistent with the classification experiments. The basic methods, including Base, All, and Random, were inferior to the embedding-based methods (including XGBoost, LightGBM, and RF) as well as the ReCoGNN method. Moreover, embedding-based methods outperformed filter-based (mutual information) and wrapper-based (backward elimination) methods across all datasets. LEVA and ARDA-NoText were superior to most baseline methods but inferior to our method.

**Summary. **To achieve better predictive performance, the key factors are: 1) adding features to the base table; and 2) ensuring that the added features are useful and do not introduce noise. By exploring the potential relationships among features within the same table and using weighted heterogeneous graphs to capture table structure, the ReCoGNN method ensures superior performance compared to other baselines.

6.3. Sensitivity of ReCoGNN

We explore the impact of varying filtering parameters $\ell$ on model performance within the MovieLens dataset, alongside the effect of $\ell$ variations. As illustrated in Fig. 6, different $\ell$ values influence model performance, depicted through a range of matrices. It is evident across these matrices that an increase in $\ell$ initially strengthens and subsequently weakens the model’s predictive capability. This occurs because, at a low $\ell$, the relational dynamics between features are inadequately captured; as $\ell$ increases, it begins to adeptly capture feature relationships within the same table. However, if $\ell$ grows too large, it may excessively capture data, thereby reducing model efficacy. Notably, when $\ell$ ranges from 0.1 to 0.4 and 0.5 to 0.7, all metrics are consistent due to the similar relationships captured by these parameter values.

As shown in Table 4, we can see that different graph discovery algorithms affect the performance of ReCoGNN on the MovieLens dataset. We compared two graph discovery algorithms, one is Complete Graph, and the other is Girvan-Newman(GN). The Girvan-Newman algorithm achieves community detection by continuously removing edges with high centrality, causing the network to gradually split into multiple tightly interconnected communities. At $\ell=$ 0.5 and 0.8, the Complete Graph algorithm performs better than the Girvan-Newman algorithm, mainly because the Complete Graph algorithm is stricter in cluster discovery, discovering strong relationships between features and thereby reducing noise.

6.4. Ablation Experiments

6.4.1. Attribute Relationship Mining.

Attribute Relationship Mining: In the preceding discussion, we emphasized the importance of mining potential relationships between attributes in relational table data. Attributes with semantic or co-dependency relationships should be grouped together to preserve such information. Splitting tuples into node groups based on these relationships allows the graph to encode group-specific patterns. However, overly aggressive splitting may fragment coherent attributes, while insufficient mining risks merging unrelated attributes, both of which could degrade the performance of downstream tasks. We designed an ablation study to evaluate the impact of attribute relationship mining. Three configurations were compared:

• •

Graph-based mining: The method mentioned in Chapter 4, which detects correlations between attributes by statistically analyzing edge weight coefficients k.

• •

Random grouping: Attributes are split into random node pairs without relationship analysis.

• •

No mining: Each attribute is treated as an independent node.

Using Accuracy and AUC-ROC as evaluation metrics, we conducted experiments on the PED and Loyal classification datasets. As shown in Figure 7, the accuracy of the feature latent relationship mining model on the relationship table is significantly better than that of randomly combined feature relationships and without feature relationship mining.

6.4.2. Edge Weights.

In section 5, we discussed how to use weight graphs to enrich and predict the features of each tuple in the base table. To examine the impact of edge weights on predictive performance, we conduct an ablation study by modifying the graph’s edge properties. In this study, we compare the results of models with and without edge weights to assess their influence on the overall accuracy. As shown in Fig. 8, in the PED and Loyal datasets, the prediction accuracy on the weighted graphs surpasses that of the methods without introduced weights; the same significant effect is observed in regression data sets. This is because GNN can learn edge weights, effectively mitigating the impact of noisy features on the final prediction during the message-passing process, thereby enhancing the overall performance of the model.

6.4.3. Graph Construction Schema.

To enhance the prediction accuracy by using relational data, we convert multi-table datasets into graphs. The primary table, known as the base table, contains nodes with labels, while the other tables provide auxiliary information to enrich the feature space of the base table. The experiment focuses on the impact of retaining or removing similarity-based edges on prediction performance. As depicted in Fig. 2, we establish two types of edges in our graph construction: (1) join relations between tables, and (2) similarity-based edges within the base table. For the similarity-based edges, each row in the base table is encoded into an embedding, and edges are added between tuples with high similarity scores. To evaluate the impact of these similarity edges, we conduct two sets of experiments: one with similarity edges included and another with them removed. From the results shown in Fig 8, it is evident that retaining similarity-based edges in the graph significantly boosts the predictive accuracy or enhances the model’s prediction precision. This is because introducing similarity edges can enhance the flow of information between nodes from the base table. More efficient information transmission can prompt the model to learn more optimized node embeddings. This greatly benefits downstream tasks such as classification and regression.

7. Related Work

Feature Augmentation Several notable works have addressed feature augmentation from relational tables. Early studies like (Kumar et al., 2016; Shah et al., 2017) avoided unnecessary joins by leveraging foreign key constraints. Later, ARDA (Chepurko et al., 2020) proposed an end-to-end automated system with heuristic feature selection, while AutoFeature (Liu et al., 2022) introduced reinforcement learning for exploration-exploitation trade-offs. Autofeat (Ionescu et al., 2024) extended this by discovering transitive features through multi-hop joins, albeit with increased computational overhead. Further improvements include METAM’s (Galhotra et al., 2023) goal-oriented discovery framework and FEATPILOT’s (Liang et al., 2025) efficient multi-hop augmentation via clustering and LSTM-based prediction. Alternatively, (Wang et al., 2022) adopted coreset-based selection to bypass table materialization while preserving accuracy.

Modeling Attribute Relationships in Tabular Data Attribute interaction methods are classified into Factorization Machines (FM) (Rendle, 2010), their enhancements, deep learning-based models, and recent GNN-based approaches. FM traditionally handle second-order feature interactions using vector inner products. Enhanced versions like Field-aware Factorization Machines (Juan et al., 2016) and Attentional Factorization Machines (Xiao et al., 2017) improve interaction modeling. Deep learning approaches, such as DeepFM (Guo et al., 2017) and xDeepFM (Lian et al., 2018), merge FM and deep networks to capture explicit and implicit high-order interactions. GNNs excel at modeling complex feature relationships; Fi-GNN (Li et al., 2019), for instance, treats features as nodes and interactions as edges for versatile attribute representation.

GNN for Tabular Data In recent research, Graph Neural Network (GNN) models have become prominent for processing relational data, expertly crafted to handle graph-structured information. These models iteratively refine node embeddings by integrating data from both the nodes and their neighbors, using these enhanced representations for predictions. Key contributions in linking relational data with GNNs are from Schlichtkrull et al. (Schlichtkrull et al., 2018), Cvitkovic et al. (Cvitkovic, 2020), and Zahradník et al. (Zahradník et al., 2023). GNN models can be categorized into inductive and transductive types based on their capability to manage unseen nodes. Inductive GNNs, such as GraphSAGE (Hamilton et al., 2017), can generalize to new nodes or graphs. In contrast, transductive GNNs like GCN (Kipf and Welling, 2016) and GIN (Xu et al., 2018) optimize node representations using the full graph but cannot adapt to new graphs. A hybrid model is GAT (Veličković et al., 2017), which supports both approaches. Our model employs the inductive strategy, allowing it to accommodate newly introduced samples in future scenarios.

8. Conclusion

In this paper,we introduce ReCoGNN, an autonomous feature augmentation framework. The framework employs a two-stage progressive column relationships discovery mechanism:(1) ReCoGNN performs parallelized modeling of latent inter-column dependencies within each auxiliary table, followed by partitioning each auxiliary table into semantically coherent segments with enhanced attribute correlations. (2) In the next stage, we construct a weighted heterogeneous graph incorporating both the base table and segmented data, enabling the GNN’s message passing to perform single-pass feature selection and base table feature augmentation. We conducted comprehensive experiments on real-world datasets to evaluate the performance of ReCoGNN against other baseline methods. The results demonstrate that ReCoGNN consistently outperforms the baselines and exhibits superior performance.