GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs

TL;DR

GaAN introduces a novel gated attention network architecture that adaptively weighs attention heads, improving performance on large, spatiotemporal graph learning tasks like node classification and traffic forecasting.

Contribution

The paper presents GaAN, a new attention mechanism with a gating sub-network, and constructs GGRU for spatiotemporal graph modeling, achieving state-of-the-art results.

Findings

GaAN outperforms existing methods on node classification.

GGRU effectively models traffic speed forecasting.

Extensive experiments validate the superiority of GaAN.

Abstract

We propose a new network architecture, Gated Attention Networks (GaAN), for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head's importance. We demonstrate the effectiveness of GaAN on the inductive node classification problem. Moreover, with GaAN as a building block, we construct the Graph Gated Recurrent Unit (GGRU) to address the traffic speed forecasting problem. Extensive experiments on three real-world datasets show that our GaAN framework achieves state-of-the-art results on both tasks.