Pointspectrum: Equivariance Meets Laplacian Filtering for Graph Representation Learning

TL;DR

PointSpectrum introduces a spectral graph representation method that combines set equivariance with Laplacian filtering to improve efficiency and address over smoothing in graph learning tasks.

Contribution

It presents a novel spectral approach that integrates set equivariant networks, enhancing spectral methods' ability to exploit graph structure and outperform existing GRL techniques.

Findings

Outperforms or matches state-of-the-art GRL methods

Addresses over smoothing in deep GNNs

Enhances spectral methods with set equivariance

Abstract

Graph Representation Learning (GRL) has become essential for modern graph data mining and learning tasks. GRL aims to capture the graph's structural information and exploit it in combination with node and edge attributes to compute low-dimensional representations. While Graph Neural Networks (GNNs) have been used in state-of-the-art GRL architectures, they have been shown to suffer from over smoothing when many GNN layers need to be stacked. In a different GRL approach, spectral methods based on graph filtering have emerged addressing over smoothing; however, up to now, they employ traditional neural networks that cannot efficiently exploit the structure of graph data. Motivated by this, we propose PointSpectrum, a spectral method that incorporates a set equivariant network to account for a graph's structure. PointSpectrum enhances the efficiency and expressiveness of spectral methods, while it outperforms or competes with state-of-the-art GRL methods. Overall, PointSpectrum addresses over smoothing by employing a graph filter and captures a graph's structure through set equivariance, lying on the intersection of GNNs and spectral methods. Our findings are promising for the benefits and applicability of this architectural shift for spectral methods and GRL.