Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks

TL;DR

This paper introduces SLiCE, a self-supervised framework that learns contextual node embeddings for link prediction in heterogeneous networks by combining global and local information without pre-defined metapaths.

Contribution

SLiCE automatically learns task-specific metapath compositions and enhances link prediction by integrating static and contextual embeddings in a self-supervised manner.

Findings

SLiCE outperforms existing static and contextual embedding methods on benchmark datasets.

The semantic association matrix improves interpretability and prediction accuracy.

Self-supervised pre-training with higher-order associations enhances downstream link prediction.

Abstract

Representation learning methods for heterogeneous networks produce a low-dimensional vector embedding for each node that is typically fixed for all tasks involving the node. Many of the existing methods focus on obtaining a static vector representation for a node in a way that is agnostic to the downstream application where it is being used. In practice, however, downstream tasks such as link prediction require specific contextual information that can be extracted from the subgraphs related to the nodes provided as input to the task. To tackle this challenge, we develop SLiCE, a framework bridging static representation learning methods using global information from the entire graph with localized attention driven mechanisms to learn contextual node representations. We first pre-train our model in a self-supervised manner by introducing higher-order semantic associations and masking nodes, and then fine-tune our model for a specific link prediction task. Instead of training node representations by aggregating information from all semantic neighbors connected via metapaths, we automatically learn the composition of different metapaths that characterize the context for a specific task without the need for any pre-defined metapaths. SLiCE significantly outperforms both static and contextual embedding learning methods on several publicly available benchmark network datasets. We also interpret the semantic association matrix and provide its utility and relevance in making successful link predictions between heterogeneous nodes in the network.