Informed, but Not Always Improved: Challenging the Benefit of Background Knowledge in GNNs

TL;DR

This paper critically examines the assumed benefits of background knowledge graphs in GNNs for biomedical tasks, revealing that their impact is often negligible and highlighting the importance of model-knowledge alignment.

Contribution

It introduces an evaluation framework to assess the true contribution of background knowledge in GNNs, demonstrating that performance gains are not guaranteed and depend on proper alignment.

Findings

GNNs with background knowledge often perform no better than simple models.

Perturbations in background knowledge have little effect on GNN performance.

Proper alignment of GNN architecture and background knowledge is crucial for potential improvements.

Abstract

In complex and low-data domains such as biomedical research, incorporating background knowledge (BK) graphs, such as protein-protein interaction (PPI) networks, into graph-based machine learning pipelines is a promising research direction. However, while BK is often assumed to improve model performance, its actual contribution and the impact of imperfect knowledge remain poorly understood. In this work, we investigate the role of BK in an important real-world task: cancer subtype classification. Surprisingly, we find that (i) state-of-the-art GNNs using BK perform no better than uninformed models like linear regression, and (ii) their performance remains largely unchanged even when the BK graph is heavily perturbed. To understand these unexpected results, we introduce an evaluation framework, which employs (i) a synthetic setting where the BK is clearly informative and (ii) a set of perturbations that simulate various imperfections in BK graphs. With this, we test the robustness of BK-aware models in both synthetic and real-world biomedical settings. Our findings reveal that careful alignment of GNN architectures and BK characteristics is necessary but holds the potential for significant performance improvements.