# Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation

## TL;DR

This paper provides a theoretical analysis of attention-based GNNs' limitations in distinguishing graph structures due to ignoring cardinality, and proposes CPA models to enhance their discriminative power, validated by experiments.

## Contribution

It identifies the failure cases of attention GNNs related to cardinality ignorance and introduces CPA models to preserve cardinality, improving expressiveness.

## Key findings

- Attention GNNs can fail to distinguish certain structures due to cardinality issues.
- CPA models improve the discriminative capacity of attention-based GNNs.
- Experimental results show CPA models outperform baseline attention GNNs.

## Abstract

Graph Neural Networks (GNNs) are powerful to learn the representation of graph-structured data. Most of the GNNs use the message-passing scheme, where the embedding of a node is iteratively updated by aggregating the information of its neighbors. To achieve a better expressive capability of node influences, attention mechanism has grown to be popular to assign trainable weights to the nodes in aggregation. Though the attention-based GNNs have achieved remarkable results in various tasks, a clear understanding of their discriminative capacities is missing. In this work, we present a theoretical analysis of the representational properties of the GNN that adopts the attention mechanism as an aggregator. Our analysis determines all cases when those attention-based GNNs can always fail to distinguish certain distinct structures. Those cases appear due to the ignorance of cardinality information in attention-based aggregation. To improve the performance of attention-based GNNs, we propose cardinality preserved attention (CPA) models that can be applied to any kind of attention mechanisms. Our experiments on node and graph classification confirm our theoretical analysis and show the competitive performance of our CPA models.