Are Graph Attention Networks Able to Model Structural Information?

TL;DR

This paper introduces GSAT, an extension of GAT that integrates structural features from random walks and graph kernels, improving graph learning by capturing higher-order topological information.

Contribution

GSAT is a novel model that combines attribute and structure-based representations, enhancing the ability of GATs to encode complex graph structures.

Findings

GSAT outperforms existing methods on graph classification tasks.

Incorporating structural features improves model accuracy.

GSAT effectively captures higher-order topological information.

Abstract

Graph Attention Networks (GATs) have emerged as powerful models for learning expressive representations from such data by adaptively weighting neighboring nodes through attention mechanisms. However, most existing approaches primarily rely on node attributes and direct neighborhood connections, often overlooking rich structural patterns that capture higher-order topological information crucial for many real-world datasets. In this work, we present the Graph Structure Attention Network (GSAT), a novel extension of GAT that jointly integrates attribute-based and structure-based representations for more effective graph learning. GSAT incorporates structural features derived from anonymous random walks (ARWs) and graph kernels to encode local topological information, enabling attention mechanisms to adapt based on the underlying graph structure. This design enhances the model's ability to discern meaningful relational dependencies within complex data. Comprehensive experiments on standard graph classification and regression benchmarks demonstrate that GSAT achieves consistent improvements over state-of-the-art graph learning methods, highlighting the value of incorporating structural context for representation learning on graphs.