Understanding When Graph Convolutional Networks Help: A Diagnostic Study on Label Scarcity and Structural Properties

TL;DR

This study investigates the conditions under which Graph Convolutional Networks (GCNs) improve semi-supervised node classification, highlighting the roles of label scarcity, feature quality, and graph homophily.

Contribution

It provides a systematic diagnostic analysis of GCN performance, revealing when and why GCNs help or hurt based on graph and feature properties.

Findings

GCNs excel under extreme label scarcity by leveraging neighborhood structure.

On highly homophilous graphs, GCNs can perform well even with random node features.

Low homophily and strong features can cause GCNs to degrade performance.

Abstract

Graph Convolutional Networks (GCNs) have become a standard approach for semi-supervised node classification, yet practitioners lack clear guidance on when GCNs provide meaningful improvements over simpler baselines. We present a diagnostic study using the Amazon Computers co-purchase data to understand when and why GCNs help. Through systematic experiments with simulated label scarcity, feature ablation, and per-class analysis, we find that GCN performance depends critically on the interaction between graph homophily and feature quality. GCNs provide the largest gains under extreme label scarcity, where they leverage neighborhood structure to compensate for limited supervision. Surprisingly, GCNs can match their original performance even when node features are replaced with random noise, suggesting that structure alone carries sufficient signal on highly homophilous graphs. However, GCNs hurt performance when homophily is low and features are already strong, as noisy neighbors corrupt good predictions. Our quadrant analysis reveals that GCNs help in three of four conditions and only hurt when low homophily meets strong features. These findings offer practical guidance for practitioners deciding whether to adopt graph-based methods.