Median activation functions for graph neural networks

TL;DR

This paper introduces median activation functions for graph neural networks, enabling better encoding of local nonlinear graph signals by considering neighborhood information, which enhances GNN capacity with minimal added complexity.

Contribution

The paper proposes median activation functions for GNNs that operate on neighborhoods, improving their ability to model local nonlinearities compared to traditional pointwise activations.

Findings

Median activation functions improve GNN performance on synthetic and real datasets.

The proposed functions add marginal complexity while enhancing capacity.

Multiresolution median activations outperform standard pointwise functions.

Abstract

Graph neural networks (GNNs) have been shown to replicate convolutional neural networks' (CNNs) superior performance in many problems involving graphs. By replacing regular convolutions with linear shift-invariant graph filters (LSI-GFs), GNNs take into account the (irregular) structure of the graph and provide meaningful representations of network data. However, LSI-GFs fail to encode local nonlinear graph signal behavior, and so do regular activation functions, which are nonlinear but pointwise. To address this issue, we propose median activation functions with support on graph neighborhoods instead of individual nodes. A GNN architecture with a trainable multirresolution version of this activation function is then tested on synthetic and real-word datasets, where we show that median activation functions can improve GNN capacity with marginal increase in complexity.