Graph Mixer Networks

TL;DR

This paper introduces Graph Mixer Networks, a new architecture inspired by MLP-Mixers, designed to improve computational efficiency and performance in graph-structured data tasks, outperforming existing Graph Transformer models.

Contribution

The paper proposes the Graph Mixer Network (GMN), applying MLP-Mixer principles to graph data, and demonstrates its superior performance over Graph Transformers.

Findings

GMN outperforms Graph Transformers in experiments.

GMN reduces computational cost compared to Transformer-based models.

Source code is publicly available.

Abstract

In recent years, the attention mechanism has demonstrated superior performance in various tasks, leading to the emergence of GAT and Graph Transformer models that utilize this mechanism to extract relational information from graph-structured data. However, the high computational cost associated with the Transformer block, as seen in Vision Transformers, has motivated the development of alternative architectures such as MLP-Mixers, which have been shown to improve performance in image tasks while reducing the computational cost. Despite the effectiveness of Transformers in graph-based tasks, their computational efficiency remains a concern. The logic behind MLP-Mixers, which addresses this issue in image tasks, has the potential to be applied to graph-structured data as well. In this paper, we propose the Graph Mixer Network (GMN), also referred to as Graph Nasreddin Nets (GNasNets), a framework that incorporates the principles of MLP-Mixers for graph-structured data. Using a PNA model with multiple aggregators as the foundation, our proposed GMN has demonstrated improved performance compared to Graph Transformers. The source code is available publicly at https://github.com/asarigun/GraphMixerNetworks.