# Invariance-Preserving Localized Activation Functions for Graph Neural Networks

## TL;DR

This paper introduces trainable, structure-aware localized activation functions for GNNs that preserve permutation invariance and enhance model capacity across various tasks.

## Contribution

It proposes graph median and max filters as nonlinear activation functions that consider graph structure, maintaining invariance and improving GNN performance.

## Key findings

- Localized activation functions improve model capacity.
- Enhanced performance in source localization, authorship attribution, recommendation, and classification.
- Modified backpropagation enables training of these functions.

## Abstract

Graph signals are signals with an irregular structure that can be described by a graph. Graph neural networks (GNNs) are information processing architectures tailored to these graph signals and made of stacked layers that compose graph convolutional filters with nonlinear activation functions. Graph convolutions endow GNNs with invariance to permutations of the graph nodes' labels. In this paper, we consider the design of trainable nonlinear activation functions that take into consideration the structure of the graph. This is accomplished by using graph median filters and graph max filters, which mimic linear graph convolutions and are shown to retain the permutation invariance of GNNs. We also discuss modifications to the backpropagation algorithm necessary to train local activation functions. The advantages of localized activation function architectures are demonstrated in four numerical experiments: source localization on synthetic graphs, authorship attribution of 19th century novels, movie recommender systems and scientific article classification. In all cases, localized activation functions are shown to improve model capacity.