Robust Graph Representation Learning via Predictive Coding

TL;DR

This paper introduces a predictive coding-based message-passing framework for graph neural networks, enhancing robustness and calibration while maintaining competitive performance in various tasks.

Contribution

It proposes a novel graph neural network model based on predictive coding principles, improving robustness and calibration without sacrificing accuracy.

Findings

Models are comparable to standard GNNs in performance.

Proposed models are better calibrated.

Models demonstrate robustness against adversarial attacks.

Abstract

Predictive coding is a message-passing framework initially developed to model information processing in the brain, and now also topic of research in machine learning due to some interesting properties. One of such properties is the natural ability of generative models to learn robust representations thanks to their peculiar credit assignment rule, that allows neural activities to converge to a solution before updating the synaptic weights. Graph neural networks are also message-passing models, which have recently shown outstanding results in diverse types of tasks in machine learning, providing interdisciplinary state-of-the-art performance on structured data. However, they are vulnerable to imperceptible adversarial attacks, and unfit for out-of-distribution generalization. In this work, we address this by building models that have the same structure of popular graph neural network architectures, but rely on the message-passing rule of predictive coding. Through an extensive set of experiments, we show that the proposed models are (i) comparable to standard ones in terms of performance in both inductive and transductive tasks, (ii) better calibrated, and (iii) robust against multiple kinds of adversarial attacks.