Implicit Graph Neural Networks

TL;DR

Implicit Graph Neural Networks (IGNN) introduce a fixed-point equilibrium approach to better capture long-range dependencies in graph data, outperforming existing GNN models across various tasks.

Contribution

This paper proposes IGNN, a novel GNN framework using implicit fixed-point equations, with theoretical guarantees and a new training method, enhancing long-range dependency modeling.

Findings

IGNN outperforms state-of-the-art GNNs on multiple tasks.

The framework effectively captures long-range dependencies.

Theoretical conditions ensure well-posedness of the model.

Abstract

Graph Neural Networks (GNNs) are widely used deep learning models that learn meaningful representations from graph-structured data. Due to the finite nature of the underlying recurrent structure, current GNN methods may struggle to capture long-range dependencies in underlying graphs. To overcome this difficulty, we propose a graph learning framework, called Implicit Graph Neural Networks (IGNN), where predictions are based on the solution of a fixed-point equilibrium equation involving implicitly defined "state" vectors. We use the Perron-Frobenius theory to derive sufficient conditions that ensure well-posedness of the framework. Leveraging implicit differentiation, we derive a tractable projected gradient descent method to train the framework. Experiments on a comprehensive range of tasks show that IGNNs consistently capture long-range dependencies and outperform the state-of-the-art GNN models.