Layer-stacked Attention for Heterogeneous Network Embedding

TL;DR

This paper introduces LATTE, a layer-stacked attention model that effectively captures higher-order relationships in heterogeneous networks, improving interpretability and achieving state-of-the-art node classification results.

Contribution

The paper proposes LATTE, a novel architecture that decomposes higher-order meta relations at each layer, enhancing interpretability and performance in heterogeneous network embedding.

Findings

LATTE achieves state-of-the-art accuracy on benchmark datasets.

The model provides interpretable insights into heterogeneous network structures.

LATTE is lightweight and effective for both transductive and inductive tasks.

Abstract

The heterogeneous network is a robust data abstraction that can model entities of different types interacting in various ways. Such heterogeneity brings rich semantic information but presents nontrivial challenges in aggregating the heterogeneous relationships between objects - especially those of higher-order indirect relations. Recent graph neural network approaches for representation learning on heterogeneous networks typically employ the attention mechanism, which is often only optimized for predictions based on direct links. Furthermore, even though most deep learning methods can aggregate higher-order information by building deeper models, such a scheme can diminish the degree of interpretability. To overcome these challenges, we explore an architecture - Layer-stacked ATTention Embedding (LATTE) - that automatically decomposes higher-order meta relations at each layer to extract the relevant heterogeneous neighborhood structures for each node. Additionally, by successively stacking layer representations, the learned node embedding offers a more interpretable aggregation scheme for nodes of different types at different neighborhood ranges. We conducted experiments on several benchmark heterogeneous network datasets. In both transductive and inductive node classification tasks, LATTE can achieve state-of-the-art performance compared to existing approaches, all while offering a lightweight model. With extensive experimental analyses and visualizations, the framework can demonstrate the ability to extract informative insights on heterogeneous networks.