Efficient Relation-aware Neighborhood Aggregation in Graph Neural Networks via Tensor Decomposition

TL;DR

This paper introduces a relation-aware graph neural network model that uses tensor decomposition to improve knowledge graph embeddings by effectively integrating relation information, achieving superior performance on benchmark datasets.

Contribution

It proposes a novel GNN encoder with tensor decomposition for relation-aware aggregation, enhancing expressiveness and efficiency in knowledge graph embedding.

Findings

Outperforms existing models on FB15k-237 and WN18RR datasets.

Uses low-rank tensor approximation for model compression and regularization.

Employs contrastive learning to handle large-scale graphs effectively.

Abstract

Numerous Graph Neural Networks (GNNs) have been developed to tackle the challenge of Knowledge Graph Embedding (KGE). However, many of these approaches overlook the crucial role of relation information and inadequately integrate it with entity information, resulting in diminished expressive power. In this paper, we propose a novel knowledge graph encoder that incorporates tensor decomposition within the aggregation function of Relational Graph Convolutional Network (R-GCN). Our model enhances the representation of neighboring entities by employing projection matrices of a low-rank tensor defined by relation types. This approach facilitates multi-task learning, thereby generating relation-aware representations. Furthermore, we introduce a low-rank estimation technique for the core tensor through CP decomposition, which effectively compresses and regularizes our model. We adopt a training strategy inspired by contrastive learning, which relieves the training limitation of the 1-N method inherent in handling vast graphs. We outperformed all our competitors on two common benchmark datasets, FB15k-237 and WN18RR, while using low-dimensional embeddings for entities and relations.