Set Twister for Single-hop Node Classification

TL;DR

This paper introduces the Set Twister, a new permutation-invariant architecture that enhances expressiveness over DeepSets, enabling better node classification accuracy with simpler, more efficient computations, potentially reducing the need for multi-hop neighborhood information.

Contribution

The paper proposes Set Twister, a novel permutation-invariant architecture that captures higher-order dependencies more effectively than DeepSets, with theoretical and empirical advantages.

Findings

Set Twister outperforms DeepSets and other GNNs in accuracy.

It maintains low computational cost and simplicity.

It can sometimes avoid multi-hop neighborhood information.

Abstract

Node classification is a central task in relational learning, with the current state-of-the-art hinging on two key principles: (i) predictions are permutation-invariant to the ordering of a node's neighbors, and (ii) predictions are a function of the node's $r$-hop neighborhood topology and attributes, $r \geq 2$. Both graph neural networks and collective inference methods (e.g., belief propagation) rely on information from up to $r$-hops away. In this work, we study if the use of more powerful permutation-invariant functions can sometimes avoid the need for classifiers to collect information beyond $1$-hop. Towards this, we introduce a new architecture, the Set Twister, which generalizes DeepSets (Zaheer et al., 2017), a simple and widely-used permutation-invariant representation. Set Twister theoretically increases expressiveness of DeepSets, allowing it to capture higher-order dependencies, while keeping its simplicity and low computational cost. Empirically, we see accuracy improvements of Set Twister over DeepSets as well as a variety of graph neural networks and collective inference schemes in several tasks, while showcasing its implementation simplicity and computational efficiency.