DiGRAF: Diffeomorphic Graph-Adaptive Activation Function

TL;DR

DiGRAF introduces a novel, graph-adaptive diffeomorphic activation function for GNNs, leveraging CPAB transformations and an additional GNN, demonstrating superior performance across diverse datasets.

Contribution

The paper presents DiGRAF, a new graph-specific activation function that is flexible, differentiable, and learned end-to-end, enhancing GNN performance.

Findings

DiGRAF outperforms traditional activation functions on multiple datasets.

The proposed method is computationally efficient and differentiable.

DiGRAF adapts effectively to various graph structures.

Abstract

In this paper, we propose a novel activation function tailored specifically for graph data in Graph Neural Networks (GNNs). Motivated by the need for graph-adaptive and flexible activation functions, we introduce DiGRAF, leveraging Continuous Piecewise-Affine Based (CPAB) transformations, which we augment with an additional GNN to learn a graph-adaptive diffeomorphic activation function in an end-to-end manner. In addition to its graph-adaptivity and flexibility, DiGRAF also possesses properties that are widely recognized as desirable for activation functions, such as differentiability, boundness within the domain, and computational efficiency. We conduct an extensive set of experiments across diverse datasets and tasks, demonstrating a consistent and superior performance of DiGRAF compared to traditional and graph-specific activation functions, highlighting its effectiveness as an activation function for GNNs. Our code is available at https://github.com/ipsitmantri/DiGRAF.