Graph-Adaptive Activation Functions for Graph Neural Networks

TL;DR

This paper introduces graph-adaptive activation functions for GNNs that incorporate feature-topology coupling, are trainable, and preserve permutation equivariance, leading to improved performance across various tasks.

Contribution

It proposes a novel class of trainable, graph-adaptive activation functions that integrate feature-topology coupling and maintain permutation equivariance in GNNs.

Findings

Enhanced performance in distributed source localization

Improved results in finite-time consensus and distributed regression

Graph-adaptive max activation functions are Lipschitz stable

Abstract

Activation functions are crucial in graph neural networks (GNNs) as they allow defining a nonlinear family of functions to capture the relationship between the input graph data and their representations. This paper proposes activation functions for GNNs that not only adapt to the graph into the nonlinearity, but are also distributable. To incorporate the feature-topology coupling into all GNN components, nodal features are nonlinearized and combined with a set of trainable parameters in a form akin to graph convolutions. The latter leads to a graph-adaptive trainable nonlinear component of the GNN that can be implemented directly or via kernel transformations, therefore, enriching the class of functions to represent the network data. Whether in the direct or kernel form, we show permutation equivariance is always preserved. We also prove the subclass of graph-adaptive max activation functions are Lipschitz stable to input perturbations. Numerical experiments with distributed source localization, finite-time consensus, distributed regression, and recommender systems corroborate our findings and show improved performance compared with pointwise as well as state-of-the-art localized nonlinearities.