Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors

TL;DR

This paper applies bifurcation theory to analyze and address oversmoothing in deep Graph Neural Networks, proposing a novel activation function approach to create stable, informative node representations.

Contribution

It introduces a bifurcation perspective to oversmoothing, analytically proves that certain activations induce stable non-homogeneous patterns, and develops a practical initialization method.

Findings

Bifurcation analysis predicts stable non-homogeneous patterns in GNNs.

Replacing monotone activations induces bifurcations that prevent oversmoothing.

Bifurcation-aware initialization improves benchmark performance.

Abstract

Graph Neural Networks (GNNs) learn node representations through iterative network-based message-passing. While powerful, deep GNNs suffer from oversmoothing, where node features converge to a homogeneous, non-informative state. We re-frame this problem of representational collapse from a \emph{bifurcation theory} perspective, characterizing oversmoothing as convergence to a stable ``homogeneous fixed point.'' Our central contribution is the theoretical discovery that this undesired stability can be broken by replacing standard monotone activations (e.g., ReLU) with a class of functions. Using Lyapunov-Schmidt reduction, we analytically prove that this substitution induces a bifurcation that destabilizes the homogeneous state and creates a new pair of stable, non-homogeneous \emph{patterns} that provably resist oversmoothing. Our theory predicts a precise, nontrivial scaling law for the amplitude of these emergent patterns, which we quantitatively validate in experiments. Finally, we demonstrate the practical utility of our theory by deriving a closed-form, bifurcation-aware initialization and showing its utility in real benchmark experiments.