Generalized Simplicial Attention Neural Networks

TL;DR

The paper introduces Generalized Simplicial Attention Neural Networks (GSANs), which extend graph neural networks to higher-order topological spaces like simplicial complexes, enabling better modeling of multi-way interactions in complex systems.

Contribution

The paper proposes a novel neural network architecture, GSANs, that processes data on simplicial complexes using self-attention mechanisms based on topological signal processing principles.

Findings

GSANs outperform existing models on various tasks.

The approach is permutation equivariant and simplicial-aware.

Effective in trajectory prediction, data imputation, and graph classification.

Abstract

Graph machine learning methods excel at leveraging pairwise relations present in the data. However, graphs are unable to fully capture the multi-way interactions inherent in many complex systems. An effective way to incorporate them is to model the data on higher-order combinatorial topological spaces, such as Simplicial Complexes (SCs) or Cell Complexes. For this reason, we introduce Generalized Simplicial Attention Neural Networks (GSANs), novel neural network architectures designed to process data living on simplicial complexes using masked self-attentional layers. Hinging on topological signal processing principles, we devise a series of principled self-attention mechanisms able to process data associated with simplices of various order, such as nodes, edges, triangles, and beyond. These schemes learn how to combine data associated with neighbor simplices of consecutive order in a task-oriented fashion, leveraging on the simplicial Dirac operator and its Dirac decomposition. We also prove that GSAN satisfies two fundamental properties: permutation equivariance and simplicial-awareness. Finally, we illustrate how our approach compares favorably with other simplicial and graph models when applied to several (inductive and transductive) tasks such as trajectory prediction, missing data imputation, graph classification, and simplex prediction.