# Graph Attention Auto-Encoders

## TL;DR

Graph Attention Auto-Encoders (GATE) are a novel neural network architecture that effectively reconstructs both node attributes and graph structure, enabling unsupervised learning on graph-structured data with competitive results.

## Contribution

Introducing GATE, a graph auto-encoder with self-attention mechanisms that reconstructs graph structure and node attributes without prior graph knowledge, supporting inductive learning.

## Key findings

- Achieves competitive node classification performance
- Outperforms some supervised baselines
- Works for both transductive and inductive tasks

## Abstract

Auto-encoders have emerged as a successful framework for unsupervised learning. However, conventional auto-encoders are incapable of utilizing explicit relations in structured data. To take advantage of relations in graph-structured data, several graph auto-encoders have recently been proposed, but they neglect to reconstruct either the graph structure or node attributes. In this paper, we present the graph attention auto-encoder (GATE), a neural network architecture for unsupervised representation learning on graph-structured data. Our architecture is able to reconstruct graph-structured inputs, including both node attributes and the graph structure, through stacked encoder/decoder layers equipped with self-attention mechanisms. In the encoder, by considering node attributes as initial node representations, each layer generates new representations of nodes by attending over their neighbors' representations. In the decoder, we attempt to reverse the encoding process to reconstruct node attributes. Moreover, node representations are regularized to reconstruct the graph structure. Our proposed architecture does not need to know the graph structure upfront, and thus it can be applied to inductive learning. Our experiments demonstrate competitive performance on several node classification benchmark datasets for transductive and inductive tasks, even exceeding the performance of supervised learning baselines in most cases.