Bilevel Graph Structure Learning, Revisited: Inner-Channel Origins of the Reported Gain

TL;DR

This paper investigates the true source of performance improvements in bilevel graph structure learning, revealing that inner training dynamics play a larger role than the graph rewiring itself, and introduces a diagnostic method to analyze this.

Contribution

It introduces frozen-$$, a control method to separate training dynamics from graph rewiring, and provides insights into their respective impacts on model performance.

Findings

Inner training dynamics account for 78-101% of the gain in flow forecasting.

On node classification, training dynamics explain 37-44% of the performance gain.

Classical spectral diagnostics can be decoupled from task-specific gains.

Abstract

Bilevel graph structure learning is widely understood to improve graph neural networks by jointly optimizing model parameters and a learned graph structure, with the resulting performance gain attributed to the rewired adjacency. We find that this attribution may be overstated: training-dynamics effects in the inner loop, rather than the rewiring itself, capture a substantial share of the gain. To establish this, we introduce frozen-$\phi$, a control that freezes the graph while retaining the inner-loop training schedule. This decomposes the bilevel gain into an inner channel of $T$-step training dynamics with implicit gradient regularization and a graph channel of the graph rewiring itself. On spatio-temporal flow forecasting the inner channel matches or exceeds the full bilevel pipeline, accounting for 78-101% of the gain; on node classification it accounts for 37-44% under a Bernoulli edge-level parameterization. We also verify that classical spectral diagnostics can dissociate from task gain. We propose frozen-$\phi$ as a standardized diagnostic for bilevel graph structure learning, with graph distillation as a method-agnostic complement. A three-precondition framework further predicts the sign of the bilevel gain on all six benchmarks.