Exploring Consistency in Graph Representations:from Graph Kernels to Graph Neural Networks

TL;DR

This paper investigates the consistency of graph representations in GNNs, compares kernel methods, and proposes a loss to enforce similarity consistency, leading to improved graph classification performance.

Contribution

It introduces a novel consistency loss for GNNs inspired by kernel analysis, bridging the gap between kernel methods and neural networks for better relational structure capture.

Findings

WLOA kernel similarities are asymptotically consistent.

Consistency across GNN layers improves graph classification.

Proposed loss enhances performance across multiple GNN architectures.

Abstract

Graph Neural Networks (GNNs) have emerged as a dominant approach in graph representation learning, yet they often struggle to capture consistent similarity relationships among graphs. While graph kernel methods such as the Weisfeiler-Lehman subtree (WL-subtree) and Weisfeiler-Lehman optimal assignment (WLOA) kernels are effective in capturing similarity relationships, they rely heavily on predefined kernels and lack sufficient non-linearity for more complex data patterns. Our work aims to bridge the gap between neural network methods and kernel approaches by enabling GNNs to consistently capture relational structures in their learned representations. Given the analogy between the message-passing process of GNNs and WL algorithms, we thoroughly compare and analyze the properties of WL-subtree and WLOA kernels. We find that the similarities captured by WLOA at different iterations are asymptotically consistent, ensuring that similar graphs remain similar in subsequent iterations, thereby leading to superior performance over the WL-subtree kernel. Inspired by these findings, we conjecture that the consistency in the similarities of graph representations across GNN layers is crucial in capturing relational structures and enhancing graph classification performance. Thus, we propose a loss to enforce the similarity of graph representations to be consistent across different layers. Our empirical analysis verifies our conjecture and shows that our proposed consistency loss can significantly enhance graph classification performance across several GNN backbones on various datasets.