Learnable Graph Convolutional Attention Networks

TL;DR

This paper introduces L-CAT, a novel GNN architecture that adaptively combines convolutional and attention-based layers, outperforming existing models across various datasets by automatically interpolating between them.

Contribution

The main contribution is the development of L-CAT, which learns to interpolate between GCN, GAT, and CAT layers using only two scalar parameters, enhancing robustness and performance.

Findings

L-CAT outperforms competing GNN methods on multiple datasets.

L-CAT reduces the need for extensive cross-validation.

L-CAT effectively combines different GNN layers for improved robustness.

Abstract

Existing Graph Neural Networks (GNNs) compute the message exchange between nodes by either aggregating uniformly (convolving) the features of all the neighboring nodes, or by applying a non-uniform score (attending) to the features. Recent works have shown the strengths and weaknesses of the resulting GNN architectures, respectively, GCNs and GATs. In this work, we aim at exploiting the strengths of both approaches to their full extent. To this end, we first introduce the graph convolutional attention layer (CAT), which relies on convolutions to compute the attention scores. Unfortunately, as in the case of GCNs and GATs, we show that there exists no clear winner between the three (neither theoretically nor in practice) as their performance directly depends on the nature of the data (i.e., of the graph and features). This result brings us to the main contribution of our work, the learnable graph convolutional attention network (L-CAT): a GNN architecture that automatically interpolates between GCN, GAT and CAT in each layer, by adding only two scalar parameters. Our results demonstrate that L-CAT is able to efficiently combine different GNN layers along the network, outperforming competing methods in a wide range of datasets, and resulting in a more robust model that reduces the need of cross-validating.