Curriculum Guided Personalized Subgraph Federated Learning

TL;DR

This paper introduces CUFL, a novel federated learning framework for subgraph GNNs that uses curriculum learning and refined client similarity measures to enhance personalization and mitigate data heterogeneity issues.

Contribution

CUFL combines curriculum learning with a new client similarity estimation method to improve personalized federated learning on subgraph data.

Findings

CUFL outperforms baseline methods on six benchmark datasets.

Curriculum learning prevents early overfitting to biased subgraph patterns.

Refined similarity measures enhance client personalization.

Abstract

Subgraph Federated Learning (FL) aims to train Graph Neural Networks (GNNs) across distributed private subgraphs, but it suffers from severe data heterogeneity. To mitigate data heterogeneity, weighted model aggregation personalizes each local GNN by assigning larger weights to parameters from clients with similar subgraph characteristics inferred from their current model states. However, the sparse and biased subgraphs often trigger rapid overfitting, causing the estimated client similarity matrix to stagnate or even collapse. As a result, aggregation loses effectiveness as clients reinforce their own biases instead of exploiting diverse knowledge otherwise available. To this end, we propose a novel personalized subgraph FL framework called Curriculum guided personalized sUbgraph Federated Learning (CUFL). On the client side, CUFL adopts Curriculum Learning (CL) that adaptively selects edges for training according to their reconstruction scores, exposing each GNN first to easier, generic cross-client substructures and only later to harder, client-specific ones. This paced exposure prevents early overfitting to biased patterns and enables gradual personalization. By regulating personalization, the curriculum also reshapes server aggregation from exchanging generic knowledge to propagating client-specific knowledge. Further, CUFL improves weighted aggregation by estimating client similarity using fine-grained structural indicators reconstructed on a random reference graph. Extensive experiments on six benchmark datasets confirm that CUFL achieves superior performance compared to relevant baselines. Code is available at https://github.com/Kang-Min-Ku/CUFL.git.

Tables

Table 1. Table 1. Node classification performance on FL frameworks over three different numbers of participating clients. The data is configured to have no overlapping edges between clients. The presented outcome is the mean and standard deviation of accuracy at the final round of training. The best result is bold , and the second result is underlined .

			Cora			CiteSeer			PubMed
Frameworks	5 Clients	10 Clients	20 Clients	5 Clients	10 Clients	20 Clients	5 Clients	10 Clients	20 Clients
Local	80.70$\pm$0.63	80.10$\pm$0.23	77.81$\pm$1.25	67.38$\pm$0.29	71.26$\pm$0.52	70.40$\pm$1.05	83.25$\pm$0.17	81.26$\pm$0.21	82.68$\pm$0.53
FedAvg	81.37$\pm$0.43	75.57$\pm$1.51	74.37$\pm$0.81	70.69$\pm$0.53	66.19$\pm$1.73	68.80$\pm$1.48	85.60$\pm$0.14	81.67$\pm$0.39	82.33$\pm$0.34
FedProx	81.13$\pm$0.31	74.12$\pm$1.87	71.37$\pm$4.23	70.91$\pm$0.78	66.22$\pm$1.71	68.90$\pm$0.98	85.59$\pm$0.13	81.23$\pm$0.60	82.22$\pm$0.36
FedAvgCL	81.97$\pm$0.75	80.52$\pm$0.38	74.88$\pm$0.84	71.36$\pm$0.65	66.65$\pm$1.07	71.72$\pm$1.62	85.98$\pm$1.04	85.08$\pm$0.41	85.49$\pm$0.46
FedPer	81.75$\pm$0.41	80.60$\pm$0.05	76.64$\pm$1.71	70.38$\pm$0.55	70.89$\pm$0.62	69.49$\pm$1.02	85.51$\pm$0.11	84.89$\pm$0.39	82.88$\pm$0.15
FedTAD	78.29$\pm$0.45	77.04$\pm$0.60	77.75$\pm$0.34	69.32$\pm$0.43	70.10$\pm$0.60	67.18$\pm$0.47	84.79$\pm$0.32	84.27$\pm$0.31	83.45$\pm$1.61
FedSpray	75.64$\pm$0.92	74.08$\pm$0.52	79.21$\pm$0.27	71.27$\pm$0.42	75.62$\pm$0.38	72.24$\pm$0.55	82.92$\pm$0.46	84.12$\pm$0.13	84.58$\pm$0.20
FedGNN	81.31$\pm$0.38	71.93$\pm$0.86	75.48$\pm$1.12	70.72$\pm$0.18	63.68$\pm$1.42	66.52$\pm$1.30	83.84$\pm$0.09	77.24$\pm$0.43	81.15$\pm$1.37
FedSage+	78.99$\pm$0.96	77.60$\pm$2.60	77.27$\pm$4.58	70.01$\pm$0.61	68.86$\pm$2.25	65.36$\pm$3.93	84.38$\pm$0.16	84.49$\pm$1.01	80.87$\pm$1.30
FedGTA	81.74$\pm$0.37	82.60$\pm$0.41	78.68$\pm$0.55	71.24$\pm$0.31	74.96$\pm$0.32	72.33$\pm$0.54	87.36$\pm$0.07	86.64$\pm$0.10	85.88$\pm$0.20
FED-PUB	81.72$\pm$0.16	81.79$\pm$0.11	77.90$\pm$1.69	71.28$\pm$0.20	72.80$\pm$1.30	70.97$\pm$1.13	85.64$\pm$0.14	85.44$\pm$0.39	85.10$\pm$0.27
CUFL (Ours)	83.66$\pm$0.19	83.93$\pm$0.25	79.35$\pm$0.38	72.04$\pm$0.19	77.24$\pm$0.22	72.82$\pm$0.54	86.54$\pm$0.06	86.22$\pm$0.05	86.07$\pm$0.19

Table 2. Table 2. Performance gain via Local Training Components

	Cora		ogbn-arxiv
Conditions	10 Clients	20 Clients	10 Clients	20 Clients
None	81.55$\pm$0.33	76.72$\pm$0.47	67.56$\pm$0.24	69.35$\pm$0.85
w/ Proximal Term	82.07$\pm$0.31	77.80$\pm$0.36	67.83$\pm$0.20	70.16$\pm$0.68
w/ CL	83.64$\pm$0.21	79.05$\pm$0.28	68.03$\pm$0.16	70.55$\pm$0.53
w/ Proximal Term, CL (Ours)	83.98$\pm$0.19	79.62$\pm$0.23	68.58$\pm$0.13	71.41$\pm$0.42

Table 3. Table 3. Performance for combinations of CL strategy and ordering mechanism. Each variant is written as CL strategy - ordering mechanism . The ordering mechanism is either a pre-trained evaluator or a random policy.

	PubMed		Amazon-Photo
CL strategies	10 Clients	20 Clients	10 Clients	20 Clients
CLNode - FedProx	84.20$\pm$0.45	83.18$\pm$1.03	91.54$\pm$0.48	86.96$\pm$0.54
IES - FedProx	85.34$\pm$0.08	85.59$\pm$0.23	92.26$\pm$0.11	88.65$\pm$0.49
CLNode - Local	80.06$\pm$1.80	69.90$\pm$1.16	90.57$\pm$0.64	86.53$\pm$0.77
IES - Local	85.01$\pm$0.17	85.24$\pm$0.25	90.61$\pm$0.14	87.05$\pm$0.38
CLNode - Random	79.98$\pm$0.34	82.98$\pm$0.34	90.18$\pm$0.05	88.50$\pm$0.28
IES - Random	78.12$\pm$0.09	83.15$\pm$0.16	89.65$\pm$0.51	86.88$\pm$0.26
IES	86.22$\pm$0.02	86.17$\pm$0.18	92.53$\pm$0.04	89.05$\pm$0.18

Table 4. Table 4. Performance variation according to the scaling factor. “adaptive” refers to the adaptive τ \tau scheduler.

	CiteSeer		Amazon-Computer
Scaling Factors ($\tau$)	10 Clients	20 Clients	10 Clients	20 Clients
3	76.76$\pm$0.72	71.53$\pm$1.47	89.72$\pm$0.08	88.60$\pm$0.11
5	77.17$\pm$0.88	71.93$\pm$1.11	89.83$\pm$0.13	88.72$\pm$0.15
10	76.37$\pm$0.41	72.17$\pm$0.96	89.61$\pm$0.09	89.36$\pm$0.12
20	75.78$\pm$0.49	71.61$\pm$1.13	88.48$\pm$0.02	87.58$\pm$0.08
adaptive	77.61$\pm$0.15	73.04$\pm$0.57	90.02$\pm$0.05	88.90$\pm$0.05

Table 5. Table 5. Dataset statistics. The number of nodes, edges, and classes for each setting is described. “Full” refers to the original global network.

		Cora				CiteSeer				PubMed
	Full	5 Clients	10 Clients	20 Clients	Full	5 Clients	10 Clients	20 Clients	Full	5 Clients	10 Clients	20 Clients
# Classes	7				6				3
# Nodes	2,485	497	248	124	2,120	424	212	106	19,717	3,943	1,971	985
# Edges	5,429	933	445	210	3,679	704	337	162	44,338	8,187	3,835	1,803
	Amazon-Computer				Amazon-Photo				ogbn-arxiv
	Full	5 Clients	10 Clients	20 Clients	Full	5 Clients	10 Clients	20 Clients	Full	5 Clients	10 Clients	20 Clients
# Classes	10				8				40
# Nodes	13,381	2,676	1,338	669	7,487	1,497	748	374	169,343	33,868	16,934	8,467
# Edges	245,861	42,240	18,067	7,816	119,043	21,569	9,660	4,273	1,166,243	205,474	91,112	43,377

Table 6. Table 6. Performance on Louvain-partitioned subgraphs.

	Cora		Amazon-Photo
Frameworks	10 Clients	20 Clients	10 Clients	20 Clients
FedProx	76.93$\pm$0.42	68.85$\pm$1.27	91.47$\pm$0.41	89.13$\pm$1.05
FED-PUB	81.29$\pm$0.23	77.79$\pm$0.39	94.15$\pm$0.21	93.20$\pm$0.15
CUFL (Ours)	82.79$\pm$0.10	79.80$\pm$0.30	95.40$\pm$0.10	95.11$\pm$0.08

Table 7. Table 7. Performance on reference graphs from different random graph models.

	Cora		PubMed
Graph Generator	10 Clients	20 Clients	10 Clients	20 Clients
Erdős–Rényi	82.10$\pm$0.25	78.76$\pm$0.56	84.36$\pm$0.04	84.50$\pm$0.29
Barabási–Albert	84.16$\pm$0.18	79.48$\pm$0.93	85.56$\pm$0.05	86.31$\pm$0.23
SBM	83.98$\pm$0.19	79.62$\pm$0.23	86.22$\pm$0.02	86.17$\pm$0.18

Table 8. Table 8. Performance on reference graphs with different random graph sizes.

	Cora		PubMed
Graph Size	10 Clients	20 Clients	10 Clients	20 Clients
$\left|\tilde{V}\right|=100$	83.61$\pm$0.33	79.25$\pm$0.42	85.93$\pm$0.06	86.03$\pm$0.23
$\left|\tilde{V}\right|=500$	83.98$\pm$0.19	79.62$\pm$0.23	86.22$\pm$0.02	86.17$\pm$0.18
$\left|\tilde{V}\right|=1000$	83.96$\pm$0.12	79.41$\pm$0.22	86.18$\pm$0.03	86.21$\pm$0.16

Equations

$$\boldsymbol{h}_{i}^{l+1}=\text{COMB}^{l}\big{(}\boldsymbol{h}_{i}^{l},\text{AGG}^{l}(\{\boldsymbol{h}_{j}^{l}:v_{j}\in\mathcal{N}(v_{i})\})\big{)},$$

$$\begin{array}{rl}& \underset{\{{\mathbf{W}}_k{\}}_{k=1}^K}{\min }\sum _{G_k\subseteq G}\mathcal{L}(G_k;{\mathbf{W}}_k),{\mathbf{W}}_k\leftarrow \sum _{n=1}^K{\alpha }_{kn}\cdot {\bar{\mathbf{W}}}_n\text{hfill}\\ & \text{with }{\alpha }_{kp}\gg {\alpha }_{kq}\text{ for }{G}_p\subseteq C\text{ and }{G}_q⊈C,\end{array}$$

$$\begin{array}{r}\underset{{\mathbf{S}}_k}{\min }\sum _{i,j}{\mathrm{S}}_k[i,j]\left(\Vert {\mathrm{A}}_k[i,j]-{\hat{\mathrm{A}}}_k^{(t)}[i,j]\Vert -{\lambda }^{(t)}\right)\\ +\frac{\gamma }{2}\Vert {\mathbf{S}}_k-{\mathbf{S}}_k^{(t)}\Vert ,\end{array}$$

$$\operatorname{Sim}(\tilde{\mathbf{u}}_{k}^{(t)}\!,\!\tilde{\mathbf{u}}_{n}^{(t)})=\frac{\bigl{\|}\tilde{\mathbf{u}}_{k}^{(t)\top}\tilde{\mathbf{u}}_{n}^{(t)}\bigr{\|}_{F}^{2}}{\bigl{\|}\tilde{\mathbf{u}}_{k}^{(t)\top}\tilde{\mathbf{u}}_{k}^{(t)}\bigr{\|}_{F}\;\bigl{\|}\tilde{\mathbf{u}}_{n}^{(t)\top}\tilde{\mathbf{u}}_{n}^{(t)}\bigr{\|}_{F}},\quad\tilde{\mathbf{u}}_{k}^{(t)}=\operatorname{ext}\bigl{(}\tilde{\mathbf{S}}_{k}^{(t)}\bigr{)},$$

$$\mathbf{W}_{k}^{(t+1)}\leftarrow\sum_{n=1}^{K}\alpha_{kn}^{(t)}\,\bar{\mathbf{W}}_{n}^{(t)},\alpha_{kn}^{(t)}=\frac{\exp\bigl{(}\tau\operatorname{Sim}(\tilde{\mathbf{u}}_{k}^{(t)}\!,\!\tilde{\mathbf{u}}_{n}^{(t)})\bigr{)}}{\sum_{p=1}^{K}\exp\bigl{(}\tau\operatorname{Sim}(\tilde{\mathbf{u}}_{k}^{(t)}\!,\!\tilde{\mathbf{u}}_{p}^{(t)})\bigr{)}},$$

$$\|\tilde{\mathbf{A}}_{k}^{(t)}-\tilde{\mathbf{A}}_{m}^{(t)}\|_{F}\;\;\leq\;\;\frac{1}{\xi}\,\bigl{\|}\tilde{\mathbf{A}}_{k}^{(t)}-\tilde{\mathbf{A}}_{n}^{(t)}\bigr{\|}_{F}\;+\;\mathcal{O}\!\bigl{(}\epsilon_{C}\bigr{)},$$

$$r_{k}^{(t)}(i,j)\;=\;\kappa\!\bigl{(}\mathbf{h}_{k,i}^{(t)},\mathbf{h}_{k,j}^{(t)}\bigr{)},$$

$${\mathbb{E}}_{i\in {\mathcal{V}}_{\mathrm{rand}}}\Vert {\mathbf{h}}_{k,i}^{(t)}-{\mathbf{h}}_{m,i}^{(t)}{\Vert }_2\le {ϵ}_C,$$

$$\underset{(i,j)\in {\mathcal{E}}_k^e}{\min }{r}_k^{(t)}(i,j)\ge \underset{(i,j)\in {\mathcal{E}}_k^h}{\max }{r}_k^{(t)}(i,j).$$

$$\|\tilde{\mathbf{A}}_{k}^{(t)}-\tilde{\mathbf{A}}_{m}^{(t)}\|_{F}\;\;\leq\;\;\frac{1}{\xi}\,\bigl{\|}\tilde{\mathbf{A}}_{k}^{(t)}-\tilde{\mathbf{A}}_{n}^{(t)}\bigr{\|}_{F}\;+\;\mathcal{O}\!\bigl{(}\epsilon_{C}\bigr{)},$$

$$\|\tilde{\mathbf{A}}_{p}^{(t)}-\tilde{\mathbf{A}}_{q}^{(t)}\|_{F}^{2}\;=\;\sum_{(i,j)}\Bigl{(}r_{p}^{(t)}(i,j)-r_{q}^{(t)}(i,j)\Bigr{)}^{2}.$$

$$\bigl{|}r_{p}^{(t)}(i,j)-r_{q}^{(t)}(i,j)\bigr{|}\;\leq\;L\,\|\mathbf{h}_{p,i}^{(t)}-\mathbf{h}_{q,i}^{(t)}\|+L\,\|\mathbf{h}_{p,j}^{(t)}-\mathbf{h}_{q,j}^{(t)}\|.$$

$$\Vert {\tilde{\mathbf{A}}}_p^{(t)}-{\tilde{\mathbf{A}}}_q^{(t)}{\Vert }_F\le \mathcal{O}({ϵ}_C).$$

$$\Vert {\tilde{\mathbf{A}}}_k^{(t)}-{\tilde{\mathbf{A}}}_m^{(t)}{\Vert }_F\le \frac{1}{\xi }\Vert {\tilde{\mathbf{A}}}_k^{(t)}-{\tilde{\mathbf{A}}}_n^{(t)}{\Vert }_F+\mathcal{O}({ϵ}_C),$$

$$\mathbb{E}\bigl{[}\mathcal{L}_{k}^{\text{test}}\bigr{]}\;\leq\;\mathcal{L}_{k}^{(t)}\;+\;c_{0}\,\sqrt{\frac{|\mathcal{E}_{k}^{e}|\;+\;\lambda^{(t)}\,|\mathcal{E}_{k}^{h}|}{|\mathcal{E}_{k}|}}\;+\;\mathcal{O}\Bigl{(}\sqrt{\tfrac{\log(1/\delta)}{|\mathcal{E}_{k}|}}\Bigr{)}.$$

$$\mathrm{Complexity}\bigl{(}\mathbf{w}\bigr{)}\;=\;\mathcal{O}\!\Bigl{(}\sqrt{\tfrac{|\mathcal{E}_{k}^{e}|+\lambda^{(t)}|\mathcal{E}_{k}^{h}|}{|\mathcal{E}_{k}|}}\Bigr{)}+\mathcal{O}\Bigl{(}\sqrt{\tfrac{\log(1/\delta)}{|\mathcal{E}_{k}|}}\,\Bigr{)}.$$

Full text

Personalized Federated Learning, Graph Neural Networks

††journalyear: 2025††copyright: cc††conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea††booktitle: Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM ’25), November 10–14, 2025, Seoul, Republic of Korea††doi: 10.1145/3746252.3761128††isbn: 979-8-4007-2040-6/2025/11††ccs: Computing methodologies Machine learning††ccs: Information systems Data mining

1. Introduction

Graph Neural Networks (GNNs) (DBLP:journals/tnn/ScarselliGTHM09, 31) have achieved significant success on structured data (DBLP:conf/www/Zhao0W24, 49, 29, 13). However, in practice, the global graph is fragmented across multiple clients, each holding a private subgraph that cannot be shared (DBLP:conf/nips/TsoyK23, 36), thus limiting every client’s view of the overall structure (DBLP:conf/nips/ZhangYLSY21, 47). Consequently, insufficient data leads to suboptimal GNN performance, highlighting the need for solutions to train performant GNNs without data disclosure. In response to this issue, Subgraph Federated Learning (FL), a distributed GNN training framework, has been proposed (liu2024federated, 25). During each training round of Subgraph FL, clients train their local GNNs, and the server then aggregates the local GNN parameters into a global model. However, due to heterogeneity in clients’ scarce and biased subgraphs, the server ends up averaging mismatched distributions, resulting in a global model that is suboptimal for many clients. To mitigate this mismatch, weighted model aggregation assigns higher weights to clients with similar data distributions, tailoring server updates to each local GNN (DBLP:conf/icml/YeNWCW23, 45, 34). Since data must stay private, client similarity—which is immediately transformed into aggregation weights—is inferred from current model states, thereby adjusting in step with the local fitting process.

Nevertheless, a key challenge persists: data sparsity and bias often induce rapid overfitting to suboptimal local patterns (DBLP:journals/corr/abs-2401-02329, 10, 24), pushing local GNNs to reinforce existing biases instead of absorbing the richer knowledge from weighted aggregation. To investigate this challenge, we first split the global graph into two partitions with METIS (karypis1997metis, 14) and then repeatedly sample 50% of the nodes from each partition, producing five partially overlapping subgraphs that share similar data properties. FedGTA (DBLP:journals/pvldb/LiWZZLW23, 23) derives client similarity from label propagation features that combine soft labels with the structural context. However, rapid overfitting (Figure 1-(A)) drives similarity scores to extremes, causing the similarity matrix to lock in prematurely (Figure 1-(B), left). With the similarity matrix frozen, subsequent updates are confined to the initial “most-similar” subgroup, causing the training process to reinforce patterns already shared inside that group. In FED-PUB (DBLP:conf/icml/BaekJJYH23, 2), client similarity is measured coarsely by comparing average node embeddings that each local GNN outputs on a shared random graph. FED-PUB likewise succumbs to collaboration lock-in due to rapid overfitting (Figure 1-(A)). The coarse functional nature of FED-PUB’s similarity estimation produces an additional failure mode whereby the matrix shrinks over successive rounds and ultimately collapses (Figure 1-(B), middle). Taken together, rapid overfitting anchors weighted aggregation in narrow local patterns, leaving little room for further enrichment on the server. Accordingly, an adaptive mechanism that regulates the degree of local model fitting is essential for breaking the lock-in and unlocking the full benefits of server aggregation.

Motivated by this goal, we integrate Curriculum Learning (CL) (DBLP:conf/icml/BengioLCW09, 4, 11, 16, 40), which organizes training samples from “easy” to “hard”, whereby easier samples remain close to data distribution, whereas harder ones embody client-specific characteristics (DBLP:conf/iccv/VahidianKB0K0S023, 37). Recognizing these insights, we propose a novel personalized Subgraph FL framework named Curriculum guided personalized sUbgraph Federated Learning (CUFL), which guides gradual personalization through a curriculum and supports it with a precise weighted aggregation method. Our CL strategy incrementally incorporates graph edges based on continuously updated difficulty, evaluated by how well the local GNNs can reconstruct these edges. Owing to their adaptivity to divergent server updates, automatic strategies furnish an appropriate curriculum for each client without additional communication or global statistics. With this design in place, each local GNN traverses a staged personalization trajectory—initially fitting to cross-client regularities in the early rounds and then gradually shifting its focus to client-specific details in later rounds. This staged fitting prevents each local GNN from overfitting to its own biased patterns too early. Beyond this, CL also reshapes the dynamics of server aggregation. Because weighted aggregation adapts to evolving local model states, the knowledge propagated by the server shifts from broadly shared patterns in the early rounds to increasingly client-specific information as training progresses (Figure 1-(B), right). However, sustaining the benefits of this evolving flow requires high-resolution estimate of client similarity. To this end, under the data constraints of FL (DBLP:conf/nips/TsoyK23, 36), client similarity is measured through comparisons of reconstructed graph structures on a shared random reference graph. Since models trained on analogous data produce similar node embedding distributions for identical inputs (DBLP:conf/icml/Kornblith0LH19, 17), their reconstructions of the reference graph are likewise comparable. These reconstructions provide fine-grained structural indicators without disclosing client data, thereby enabling high-resolution and privacy-preserving similarity estimation. Comprehensive experiments on six benchmark datasets demonstrate outstanding performance of CUFL. Further analyses show the impact of CL on local GNN personalization as well as the robustness of CUFL’s similarity estimation method.

2. Related Works

2.1. Subgraph Federated Learning

Federated Learning (FL) (DBLP:journals/mlc/WenZLCCZ23, 41) often struggles in subgraph tasks by overlooking structural characteristics, motivating subgraph FL frameworks for the distributed subgraph setting (liu2024federated, 25).

Several frameworks transfer knowledge between global and local models. FedSpray (DBLP:conf/kdd/FuCZ0L24, 8) distills unbiased soft labels from a global feature-structure encoder to local models. FedGF (DBLP:conf/wsdm/Zhou00LYFLWZW25, 50) employs graph atoms with personalized weights to inject customized global structural knowledge into local models. While these frameworks enhance local models with global information, CUFL improves local GNNs without global guidance. In the opposite direction, FedTAD (DBLP:conf/ijcai/ZhuLWWHL24, 51) uploads class-specific knowledge from local GNNs to the global model according to each client’s reliability for that class.

Alternatively, another line of research focuses on expanding local subgraphs. FedGNN (DBLP:journals/corr/abs-2102-04925, 42) extends overlapping nodes from other subgraphs. In a related manner, FedSage+ (DBLP:conf/nips/ZhangYLSY21, 47) generates missing neighbors and mends the local subgraph by incorporating them. However, these subgraph augmentation methods suffer from unintended data exposure and communication overhead.

Weighted model aggregation, which tailors collaboration to each client, has also been studied. FedSG (DBLP:journals/tetci/WangGQLL24, 38) measures client similarity by separately comparing the parameters of each component in the local model, which is computationally inefficient. FedGTA (DBLP:journals/pvldb/LiWZZLW23, 23) groups clients from mixed moments of neighbor features obtained via label propagation. However, rapid overfitting locks server aggregation into peer groups, curbing the intake of diverse, beneficial knowledge. In a separate line of work, FED-PUB (DBLP:conf/icml/BaekJJYH23, 2) points out data heterogeneity induced by community structure and proposes a solution by evaluating the similarity of functional embeddings on a random graph to predict communities. Under rapid overfitting, client similarity estimation of FED-PUB gradually collapses as training progresses.

2.2. Curriculum Graph Learning

Curriculum Learning (CL) comprises three components. A score function assigns difficulty, a scheduling function orders training samples, and a pacing function controls the pace of data presentation. CL strategies are categorized by whether the learning order changes continuously during training. Pre-defined strategies use fixed learning orders, while automatic strategies adaptively adjust them during training by accepting feedback from the model (DBLP:conf/ijcai/LiW023, 21).

Among CL strategies designed for graph data, CLNode (DBLP:conf/wsdm/WeiGZ00H23, 39) utilizes a pre-trained GNN to pre-define node difficulty, considering both node labels and features. On the other hand, RCL (DBLP:conf/nips/ZhangWZ23, 48) integrates confident edges incrementally based on the current model expectations. In addition to strategies, the effectiveness of CL has been widely studied both empirically and theoretically. (DBLP:conf/icml/BengioLCW09, 4, 16, 40, 11) reveal that CL contributes to better generalization, robust optimization, and faster convergence. Furthermore, (DBLP:conf/icml/HacohenW19, 11) proves its ability to subtly steepen the optimization landscape without altering the global optimum. Following these insights, (DBLP:conf/iccv/VahidianKB0K0S023, 37) demonstrates that CL alleviates data heterogeneity in FL. The study finds that CL is particularly useful during early training rounds, as the “easier” samples tend to be closer in distribution. While their work illustrates the relationship between CL and FL, it is limited to centralized FL.

3. Preliminaries

Given $K$ clients, each client $k$ owns a local subgraph $G_{k}=(\mathcal{V}_{k},\mathcal{E}_{k})$, which constitutes a substructure of the global graph $G=(\mathcal{V},\mathcal{E})$, where $\mathcal{V}_{k}\subseteq\mathcal{V}$ and $\mathcal{E}_{k}\subseteq\mathcal{E}$. $\mathcal{V}_{k}$ refers to a set of $|\mathcal{V}_{k}|$ nodes, and $\mathcal{E}_{k}$ to a set of $|\mathcal{E}_{k}|$ edges. Client $k$’s $i$-th node $v_{k,i}\in\mathcal{V}_{k}$ is represented by a feature vector $\boldsymbol{x}_{k,i}\in\mathbb{R}^{d_{\boldsymbol{x}}}$, where $d_{\boldsymbol{x}}$ indicates its dimensionality. The edges in $G_{k}$ are encoded in its adjacency matrix $\mathbf{A}_{k}\in\{0,1\}^{|\mathcal{V}_{k}|\times|\mathcal{V}_{k}|}$. Specifically, $\mathrm{A}_{k}[i,j]=1$ if a link exists between the $i$-th and $j$-th nodes in $\mathcal{V}_{k}$, otherwise $\mathrm{A}_{k}[i,j]=0$.

3.1. Graph Neural Networks

Graph Neural Networks (GNNs) typically follow a message-passing framework (DBLP:conf/icml/GilmerSRVD17, 9), iteratively aggregating and combining node representations to refine each node. GNNs express an $i$-th node $v_{i}$ at the $l$-th layer as follows (the client index $k$ is omitted for clarity):

[TABLE]

where $\boldsymbol{h}_{i}^{l}$ denotes the representation of $v_{i}$ at the $l$-th layer. The representation is initialized as $\boldsymbol{h}_{i}^{0}=\boldsymbol{x}_{i}$, and the final output is described as $\boldsymbol{h}_{i}$. $\mathcal{N}(v_{i})$ indicates the set of neighbors of $v_{i}$. $\text{AGG}^{l}$ aggregates representations of $\mathcal{N}(v_{i})$, and $\text{COMB}^{l}$ combines the previous representation of $v_{i}$ with the aggregated representation.

3.2. Personalized Subgraph FL Optimization

In Subgraph Federated Learning (FL), data heterogeneity hinders the effectiveness of a one-size-fits-all model, which often yields suboptimal performance for some clients. In contrast, personalizing GNNs to each client’s objective mitigates this heterogeneity. The weighted model aggregation facilitates personalization by promoting parameter sharing among mutually beneficial peers. The resulting personalized Subgraph FL objective (DBLP:conf/icml/BaekJJYH23, 2) is formulated as:

[TABLE]

where $\mathcal{L}$ denotes the task-specific loss, $\mathbf{W}_{k}$ the aggregated parameters for client $k$, $\bar{\mathbf{W}}_{k}$ the locally trained parameters for client $k$, and $C$ a cluster of clients with similar data distributions. $\alpha_{kn}$ is the coefficient for weighted aggregation between clients $k$ and $n$, which takes higher values for clients with subgraphs that are part of the same cluster.

4. CUFL Framework

4.1. Local Training Stage

To enhance the benefits of weighted model aggregation, we employ CL, which flexibly tunes personalization throughout local training.

4.1.1. Incremental Edge Selection (IES)

We adopt Incremental Edge Selection (IES) (DBLP:conf/nips/ZhangWZ23, 48) for dynamically generating personalized edge masks based on the understanding of the current model. This automatic CL strategy fits well with personalized Subgraph FL, as it remains adaptable to divergent server updates without additional communication or global statistics.

The objective of the IES module for client $k$ is to optimize the learnable mask matrix $\mathbf{S}_{k}\in\mathbb{R}^{|\mathcal{V}_{k}|\times|\mathcal{V}_{k}|}$, which refines the local subgraph structure. Specifically, at round $t$, the adjacency matrix of the input graph $G_{k}^{(t)}$ is derived as $\mathbf{S}_{k}^{(t)}\odot\mathbf{A}_{k}$, where $\odot$ indicates the Hadamard product, and $\mathbf{A}_{k}$ represents the adjacency matrix of the local subgraph $G_{k}$. The difficulty of edges in IES is inversely proportional to the similarity between the current embeddings of their connected nodes. As illustrated in Figure 2-(A), the node embedding matrix $\mathbf{H}_{k}^{(t)}$, whose $i$-th row $\boldsymbol{h}_{k,i}^{(t)}$ corresponds to the output embedding of the $i$-th node extracted from the local GNN using the refined local subgraph, is assembled. Then, the reconstructed graph $\hat{G}_{k}^{(t)}$ is produced through $\mathbf{H}_{k}^{(t)}$. In the adjacency matrix $\hat{\mathbf{A}}^{(t)}_{k}$ of $\hat{G}_{k}^{(t)}$, the edge weight between nodes $i$ and $j$ is defined as $\kappa(\mathbf{H}^{(t)}_{k}[i,:],\mathbf{H}^{(t)}_{k}[j,:])$, where $\kappa$ is a kernel function implemented as cosine similarity. IES prioritizes “well-expected” edges with higher weights of $\hat{G}_{k}^{(t)}$ for inclusion in training.

IES controls the pace of local training with an aging parameter $\lambda^{(t)}$. That is, $\lambda^{(t)}$ quantitatively determines which edge qualifies as a “well-expected” edge. $\lambda^{(t)}$ is defined as $g_{\lambda}(t)=\min(\frac{\zeta}{R}t,1)$, where $R$ is the total number of rounds and $\zeta$ designates the round when the full local subgraph is introduced. The overall loss is denoted as:

[TABLE]

where $\gamma$ is a regularization coefficient and $\mathbf{S}_{k}^{(t)}$ indicates the current mask matrix. The first term increases the mask weights for edges whose residual errors exceed $\lambda^{(t)}$. Among them, easier edges with smaller residual errors show faster growth in mask weights, leading to their earlier inclusion in training. The second term regularizes the mask to prevent abrupt changes, providing a smooth curriculum. Hence, IES initiates training with easier substructures and then incrementally admits harder substructures. At the outset of training, local GNNs predominantly fit low-frequency spectral components, which encode generic structural knowledge shared across clients (DBLP:conf/icml/RahamanBADLHBC19, 30, 35, 46). Consequently, early-round substructures exhibit highly similar distributions across clients, and as the curriculum progressively releases client-specific edges, each model personalizes gradually. This staged fitting schedule suppresses rapid overfitting by guiding each model from shared patterns to individualized patterns. Furthermore, by altering local training, the curriculum reconfigures server aggregation—initial rounds share generic knowledge, while subsequent rounds deliver more personalized updates.

4.1.2. Local Model Optimization

In the first round, all local GNNs are initialized with identical weights. We then warm up client $k$’s subgraph mask $\mathbf{S}_{k}^{(1)}$ in $\text{IES}_{train,k}$ using a GNN pre-trained with FedProx (DBLP:conf/mlsys/LiSZSTS20, 22). This yields a cohesive initialization that keeps early-round mask updates aligned across clients (DBLP:conf/iclr/NguyenWMSR23, 28).

During local training, the local GNN and the subgraph mask are optimized alternately (Figure 2-(B)). Algorithm 1 first applies the current mask $\mathbf{S}_{k}^{(t)}$ to the local subgraph $G_{k}$ (Line 3), and then trains the local GNN on the refined subgraph $G_{k}^{(t)}$ (Line 4). Complementing CL, we add the proximal term (DBLP:conf/mlsys/LiSZSTS20, 22) to the training loss, further constraining the magnitude of local updates. After training the local GNN, the latent node embedding matrix $\mathbf{H}_{k}^{(t)}$ is computed (Line 5), and the adjacency matrix is reconstructed using $\mathbf{H}_{k}^{(t)}$ (Line 6). Next, $\mathbf{S}^{(t)}_{k}$ of $\text{IES}_{train,k}$ is optimized (Lines 7-8). The $\operatorname{clip}(\cdot)$ function keeps all edge weights in the interval $[0,1]$. Finally, the subgraph mask $\mathbf{S}_{k}^{(t+1)}$ and the aging parameter $\lambda^{(t+1)}$ are stored on the client for the next round (Line 10).

4.2. Server Aggregation Stage

The success of recent personalized FL frameworks (DBLP:conf/icml/YeNWCW23, 45, 34) underscores the efficacy of weighted model aggregation, which lets each client receive favorable parameters from helpful peers while avoiding detrimental ones. In this section, we present a method for deriving fine-grained signals that precisely estimate client similarity, and examine how CL reconfigures weighted model aggregation.

4.2.1. Client Similarity Estimation via Node Embedding Distributions

The intuition behind the proposed method is that local GNNs trained on subgraphs with similar properties exhibit comparable node embedding distributions for the same input (DBLP:conf/icml/Kornblith0LH19, 17). Such similarity in node embedding distributions arises as these local GNNs converge toward similar directions (DBLP:conf/nips/CunKS90, 18). Consider two data-similar clients $k$ and $n$. For the same input, node embeddings close in the embedding space of $k$ remain close in the embedding space of $n$. This consistency is also preserved when reconstructing the graph from node embeddings, resulting in reconstructed graphs of $k$ and $n$ having analogous structures. Building on this, we feed a shared random graph to all clients. After each reconstructs it with its local GNN, we compare the reconstructed structures to evaluate client similarity. By indirectly comparing embedding distributions through the reconstructed graphs, our similarity estimation approach operates at a fine-grained level. At the same time, as the random graph is generated regardless of local subgraphs, the reconstruction-based comparison safeguards data privacy. The random graph $\tilde{G}=(\tilde{\mathcal{V}},\tilde{\mathcal{E}})$ is generated with a stochastic block model (DBLP:journals/ans/LeeW19, 19) because reconstructing multiple blocks offers richer structural cues than a single block. Each node feature is initialized from a Gaussian distribution. The random graph is reconstructed using its latent node embeddings from the local GNN, with edge weights computed based on the cosine similarity between embeddings of connected nodes.

To avoid redundant or noisy comparisons, we selectively compare the “well-expected” substructures, identified through the additional $\text{IES}_{aggr}$ module. More precisely, we employ the subgraph mask $\tilde{\mathbf{S}}_{k}$ of $\text{IES}_{aggr,k}$, optimized for the random graph via Equation (3), as the indicator in client similarity estimation (Figure 2-(C)). This approach not only ensures comparisons on “well-expected” substructures but also preserves the continuity of client similarity estimation. The similarity index between arbitrary client $k$ and $n$ at round $t$ is measured using Linear CKA (DBLP:conf/icml/Kornblith0LH19, 17), expressed as:

[TABLE]

where $\tilde{\mathbf{S}}_{k}^{(t)}\in\mathbb{R}^{|\tilde{\mathcal{V}}|\times|\tilde{\mathcal{V}}|}$ denotes the mask for the random graph of client $k$ at round $t$. The $\operatorname{ext}(\cdot)$ function extracts all edge weights from the mask matrix and converts them into a $|\tilde{\mathcal{E}}|$-dimensional vector, subsequently setting the lowest percentiles of values to zero. This vectorization can improve computational efficiency, while weight pruning helps preserve clearer “well-expected” substructures. The effectiveness of our client similarity index stems from its edge-wise multiplication, which assigns higher similarity to pairs with closely matching reconstructed random graph structures.

Toy Experiment.

To illustrate how well such reconstructions capture client similarity, we split the Cora dataset into two partitions using METIS (karypis1997metis, 14), twice sample 50% of nodes from the first to create overlapping subgraphs for Clients 1 and 2, and sample an equal-size subgraph from the second partition as Client 3. Each client’s local GNN then reconstructs the same random graph by computing an adjacency matrix based on the cosine similarities among its learned node embeddings. Figure 3 quantifies the structural alignment between reconstructed graphs by using Client 1 as reference. Client 1’s reconstructed edges are first grouped into five weight bins. For each of Clients 2 and 3, a bin-match is counted whenever the corresponding edge falls into the same bin as in Client 1. The match count in each bin is normalized by Client 1’s edge total for that bin, so a higher ratio means stronger edge-wise agreement in that bin with the reference reconstruction. Empirical results show that clients with similar data produce more closely matched reconstructed random graph structures than clients with dissimilar data. Notably, data-homogeneous clients share a substantial portion of their “well-expected” edges. Accordingly, we primarily focus on the “well-expected” substructures to refine client similarity estimation.

4.2.2. Personalized Parameter Aggregation

We now update local GNN parameters through weighted aggregation guided by client similarity. The parameters are aggregated as follows:

[TABLE]

where $\mathbf{W}_{k}^{(t+1)}$ represents the aggregated model parameters, $\bar{\mathbf{W}}_{n}^{(t)}$ denotes the locally trained model parameters, $\alpha_{kn}^{(t)}$ is the normalized similarity between clients $k$ and $n$, and $\tau$ is a scaling factor determining the intensity of collaboration. This personalized aggregation encourages cooperation among clients sharing similar data, whose local GNNs behave in a similar manner. In CUFL, the aggregation procedure is further recalibrated through CL. In the initial rounds, parameter sharing is broad because the curriculum begins with “easier” samples drawn from closely aligned data distributions. As each local GNN becomes more personalized, the aggregation assigns relatively smaller weights to dissimilar clients while proportionally increasing the weight of data-aligned clients.

4.3. Complexity Analysis

In this section, we provide a complexity analysis of our proposed CUFL. In the local training stage, the main additional cost arises from the incorporation of CL. The computational cost of learning the subgraph mask is $\mathcal{O}(|\mathcal{E}_{k}|\cdot|h_{k,i}|+|\mathcal{E}_{k}|)$, where $\mathcal{E}_{k}$ is the edge set of the local subgraph and $h_{k,i}$ denotes the output embedding of the $i$-th node produced by the local GNN. Since the complexity of optimizing the subgraph mask is comparable to or even lower than that of the local GNN training (DBLP:journals/corr/abs-1901-00596, 43), it introduces only a marginal computational burden. In the server aggregation stage, constructing a fine-grained client indicator requires $\mathcal{O}(|\tilde{\mathcal{E}}|\cdot|h_{k,i}|+|\tilde{\mathcal{E}}|)$, where $\tilde{\mathcal{E}}$ is the edge set of the random graph. As for communication overhead, each client transmits the mask of the random graph to the server, incurring a cost of $\mathcal{O}(|\tilde{\mathcal{E}}|)$. In CUFL, the overall cost of server aggregation depends on the size of the random graph $\tilde{G}$, which is generally much smaller than the local subgraph. Such independence from the local subgraph size (DBLP:journals/pvldb/LiWZZLW23, 23) and the model parameter size (DBLP:journals/tetci/WangGQLL24, 38) ensures that CUFL remains scalable.

4.4. Theoretical Analysis

In this section, we present a theoretical analysis that guarantees data-similar clients retain similar adjacency reconstructions for the same random graph, while remaining distinct from data-dissimilar clients. This statement relies on standard assumptions of Lipschitz continuity for the reconstructability function and bounded embedding divergence for data-similar clients.

Theorem 1 (Cluster Preservation).

Suppose clients $k$ and $m$ belong to the same cluster $C$ of similar data properties, whereas client $n$ does not. Let $\tilde{\mathbf{A}}_{k}^{(t)}$, $\tilde{\mathbf{A}}_{m}^{(t)}$, and $\tilde{\mathbf{A}}_{n}^{(t)}$ be their reconstructed adjacency matrices at round $t$ for the random graph. Under the Lipschitz property of $\kappa$ and the coherence assumption of Equation (7) in Appendix B, there exists $\xi>0$ such that

[TABLE]

for some small $\epsilon_{C}>0$. Hence, same-cluster reconstructions remain closer than cross-cluster reconstructions.

The theorem implies that, with high probability, data-similar clients generate closely matching reconstructions of a (potentially global) random graph, whereas this correspondence fails for clients whose data differ. Such a gap ensures that our method—when measuring similarity across clients—can effectively identify helpful peers and preserve their structure over the course of training. The full proof and a more detailed discussion appear in Appendix B.

5. Experiments

5.1. Experimental Setups

5.1.1. Datasets

We simulate a distributed subgraph environment using METIS (karypis1997metis, 14), a non-overlapping partitioning algorithm that allows explicit control of the number of subgraphs for a well-structured experiments. Experiments are conducted under the transductive setting, where nodes within each subgraph are divided into train, validation, and test sets in a 2:4:4 ratio. To assess performance, We use six benchmark datasets: Cora, CiteSeer and PubMed, small citation graphs (DBLP:conf/icml/YangCS16, 44); Amazon-Computer and Amazon-Photo, product graphs (DBLP:journals/corr/abs-1811-05868, 32); ogbn-arxiv, a large citation graph (DBLP:conf/nips/HuFZDRLCL20, 12).

5.1.2. Baselines

We compare CUFL with Local, a baseline with local training only. For general FL, we perform comparisons on FedAvg (DBLP:conf/aistats/McMahanMRHA17, 26), a centralized FL baseline that performs weighted aggregation based on local dataset size; FedProx (DBLP:conf/mlsys/LiSZSTS20, 22), that adds a proximal term to FedAvg; FedAvgCL, that applies the same CL strategy as CUFL to FedAvg; and FedPer (DBLP:journals/corr/abs-1912-00818, 1), a personalized FL baseline that separates the model into shared and personalized layers. For Subgraph FL, we employ FedTAD (DBLP:conf/ijcai/ZhuLWWHL24, 51) and FedSpray (DBLP:conf/kdd/FuCZ0L24, 8), that leverage knowledge transfer; FedGNN (DBLP:journals/corr/abs-2102-04925, 42) and FedSage+ (DBLP:conf/nips/ZhangYLSY21, 47), for data augmentation methods; and FedGTA (DBLP:journals/pvldb/LiWZZLW23, 23) and FED-PUB (DBLP:conf/icml/BaekJJYH23, 2), customizing collaboration for individual clients.

5.1.3. Hyperparameters

The local learner is a two-layer Graph Convolution Network (DBLP:conf/iclr/KipfW17, 15) with a hidden dimension size of 128. Models are trained for 100 rounds with 1 epoch on citation graphs, and for 200 rounds with 2 and 3 epochs on product graphs and ogbn-arxiv, respectively. The local learning rate is chosen by grid search in {0.01, 0.001}, while $\text{IES}_{train}$ and $\text{IES}_{aggr}$ use rates of 0.0005 and 0.00001, respectively. We set the coefficient for the proximal term $\beta$ and IES regularizer $\gamma$ to 0.001; adopt Adam for local optimization. The pacing factor is $\zeta=1.5$, and in $ext$ lowest 30% values are set to zero. The scaling factor $\tau$ is selected via grid search over {5, 10, adaptive $\tau$ scheduler}, where the adaptive $\tau$ scheduler greedily adjusts each client’s $\tau$ for the next round using the performance of the previous round (subramanian2023zeroth, 33). We generate a random graph using the stochastic block model with five blocks of 100 nodes each, where node features are drawn from $\mathcal{N}(0,1)$. Edges are added with probability 0.1 within partitions and none across partitions.

5.2. Main Results

5.2.1. Performance and Effectiveness

Table 1 reports node classification results in the node non-overlapping scenario, where clients inherently cluster based on similar data characteristics. CUFL consistently outperforms all baselines in nearly every setting. Compared to FedTAD and FedSpray, CUFL achieves higher average performance by at least (2.99%, 3.50%, 2.04%) for 5, 10, and 20 clients, respectively. CUFL also surpasses augmentation methods such as FedGNN and FedSage+, yielding gains of at least (2.26%, 6.70%, 4.38%). The performance of data augmentation approaches drops as participants grow because missing edges proliferate. By contrast, CUFL scales gracefully as its curriculum guided personalization is executed entirely on the client. Within the class of weighted model aggregation frameworks, CUFL posts the best results, outperforming FedGTA and FED-PUB by minimum margins of (0.72%, 1.58%, 1.09%). FedGTA shows strong results when label propagation is favorable, as on PubMed with 5 clients. However, it underperforms on datasets like ogbn-arxiv, where label propagation is less informative. Unlike methods relying on dataset-specific assumptions, the server aggregation in CUFL remains performant across datasets.

Alongside delivering exceptional performance, CUFL demonstrates efficient learning. As Figure 4 indicates, the curriculum schedule lets CUFL achieve fast convergence despite pronounced heterogeneity (DBLP:conf/icml/HacohenW19, 11). Moreover, the narrow performance spread in Figure 4 highlights the stability of the training process. This stable behavior, attributed to the gradual personalization enabled by CL, results in relatively low standard deviations, as shown in Table 1.

5.2.2. Impact of CL on Server Aggregation

To examine how CL reshapes server aggregation process, we track the proportion of aggregation weight assigned to clients with similar data traits. Figure 5 plots this proportion, defined as $\tfrac{1}{K}\sum\limits_{k=1}^{K}\tfrac{\sum_{n\in C_{k}}\operatorname{Sim}(\tilde{\mathbf{u}}_{k}^{(t)},\tilde{\mathbf{u}}_{n}^{(t)})}{\sum_{n=1}^{K}\operatorname{Sim}(\tilde{\mathbf{u}}_{k}^{(t)},\tilde{\mathbf{u}}_{n}^{(t)})}$, where $K$ is the total number of clients, $C_{k}$ is the set of clients whose data properties are similar to client $k$, $\operatorname{Sim}(\cdot,\cdot)$ denotes inter-client similarity, and $\tilde{\mathbf{u}}_{k}^{(t)}$ is client $k$’s fine-grained information at round $t$. FedGTA maintains the proportion almost constant. Rapid overfitting drives the low variability in that proportion, ultimately making local GNNs reinforce biased patterns. For both FED-PUB and CUFL without CL, the proportion is initially high and even peaks in the early rounds. As training continues, it drops while dissimilar clients gain more weight, eroding the intended collaboration. In FED-PUB, the coarse functional similarity induces an abrupt decline in the proportion and amplifies its oscillations. With CL enabled, the proportion begins modest, reflecting that local GNNs initially fit generic structural knowledge; as harder samples arrive, it rises smoothly, adapting the models to client-specific patterns.

5.3. Ablation Studies

5.3.1. Regularization in Local Training Stage

To prevent rapid overfitting at each client, CUFL regularizes the local GNN with a proximal term (DBLP:conf/mlsys/LiSZSTS20, 22) that restrains local updates from diverging too far, and regularizes the local subgraph through CL. Table 2 shows that each mechanism alone raises accuracy and lowers variance, with CL providing the larger gain. When combined, the proximal term and CL yield the most effective performance, confirming that their synergy is crucial for regulating the personalization of local GNNs.

5.3.2. Effectiveness of Automatic CL Strategy

CUFL refines its curriculum on-the-fly from local GNN feedback. To evaluate the effectiveness of this adaptive design, we evaluate CUFL with two CL strategies, CLNode (DBLP:conf/wsdm/WeiGZ00H23, 39) and IES, and combine each strategy with three ordering policies: (i) a global model trained with FedProx, (ii) a local model trained only on the local subgraph (Local), and (iii) a uniformly random schedule (Random). (i) and (ii) are pre-defined curricula; (iii) is order-agnostic. As reported in Table 3, the pre-defined curricula perform noticeably worse in personalized Subgraph FL. We attribute this gap to the widening mismatch between the evolving local GNNs and the fixed teacher models, which are difficult to keep sufficiently mature in a distributed subgraph environment. Furthermore, the automatic curricula also surpass the random baseline. The uniformly random schedule, being blind to model feedback, fails to achieve the gradual personalization essential to effective local GNN training.

5.3.3. Edge Filtering Approaches for Client Similarity Estimation

This part corroborates that “well-expected” edges in reconstructed graphs are appropriate for computing client similarity. To this end, we compare similarity matrices under three settings—all edges, thresholding, and IES—in the node overlapping scenario. Thresholding removes edges with weights below a preset threshold of 0.5. As shown in Figure 6-(A), the contrast among estimated groups of clients with distinct data characteristics becomes blurred once all edges are retained. To elaborate further, the output embeddings from the local GNN for inputs derived from a single distribution tend to be smoothed (DBLP:journals/corr/abs-1911-08795, 7, 20). This smoothing results in a reconstructed random graph with a significant portion of its edge weights at moderate values, causing substantial overlaps in the graph structures from most clients. Thresholding can treat the problem of indistinct weighted model aggregation (Figure 6-(B)), but defining a suitable threshold remains challenging. In contrast, optimizing IES enables pronounced weighted model aggregation without requiring comprehensive hyperparameter tuning (Figure 6-(C)).

5.3.4. Varying Scaling Factor $\tau$

Because heterogeneity can change markedly according to data, the success of weighted model aggregation depends on considering both accurate client similarity estimation and the appropriate scaling of collaboration intensity. Yet, while extensive attention has been devoted to estimating client similarity, the proper intensity of collaboration is unexplored. Therefore, we investigate it by evaluating CUFL’s performance across various scaling factors. Alongside fixed $\tau$, we implement an adaptive $\tau$ scheduler that greedily adjusts each client’s $\tau$ to its performance change relative to the previous round (subramanian2023zeroth, 33). As illustrated in Table 4, a small $\tau$ (i.e., $\tau=3$) produces sub-optimal outcomes, causing insufficient cooperation among clients with similar data distributions. The adaptive $\tau$ scheduler steadily delivers superior performance by providing each client with a tailored scaling factor. When excluding this scheduler, the appropriate $\tau$ is 5 for 10 clients and 10 for 20 clients. The trend implies that rising client counts exacerbate data heterogeneity, so directing communication toward the most relevant peers yields the greatest benefit. However, when aggregation becomes overly inward-looking (i.e., $\tau=20$), performance paradoxically degrades, because of the absence of outward updates that could otherwise help navigate obstacles in the loss landscape.

6. Conclusion

In this work, we tackle the overlooked problem of rapid overfitting in weighted model aggregation. Rapid overfitting constrains the benefits of selective collaboration because, during server aggregation, local GNNs intensify their biased patterns rather than assimilate complementary knowledge. To counter this, we introduce CUFL, which combines automatic CL on each client with robust weighted model aggregation on the server. The approach steers local GNNs toward gradual personalization and shifts server aggregation from exchanging generic knowledge to prioritizing client-specific insights. Experimental results demonstrate that CUFL generally outperforms existing FL frameworks, showing outstanding performance across various benchmark datasets. Further analyses emphasize the impact of CL on server aggregation, the suitability of our adaptive CL strategy for personalized Subgraph FL, and the proper intensity of collaboration.

Acknowledgements

This work was supported by the Institute of Information & Communications Technology Planning and Evaluation (IITP) and the National IT Industry Promotion Agency (NIPA) through grants funded by the Korean government (MSIT) (RS-2019-II190421, IITP-2025-RS-2020-II201821, RS-2025-02218768, RS-2025-25443718, RS-2025-25442569, RS-2025-02653113, H0601-24-1023). ChatGPT is used for writing (English refinement, typo correction) and code (bug-fix suggestions); all AI outputs are reviewed and edited by the authors.

Appendix A Server Aggregation Stage Algorithm

After the local training stage, each client uploads the parameters of the local Graph Neural Network (GNN) to the server. In Algorithm 2, the set $\mathbb{S}$ containing the mask matrices of each local GNN for the random graph is initialized (Line 1). The central server optimizes the mask matrix for all local GNNs on the random graph (Lines 2-6). Subsequently, the local GNN parameters are aggregated based on Equations (4) and 5 (Lines 7-8). Note that $\tilde{\mathbf{S}}^{(t)}_{k}$ can be represented as a vector of length $|\tilde{\mathcal{E}}|$ where $|\tilde{\mathcal{E}}|$ is number of edges in random graph $\tilde{G}$. This vectorization offers a more efficient approach to determining client similarity rather than operating on matrices.

Appendix B Detailed Theoretical Analysis

We provide a theoretical analysis of (i) the robustness of CUFL’s client similarity estimation method and (ii) how Incremental Edge Selection (IES) enhances overfitting mitigation in federated learning over subgraphs. Section B.1 introduces key assumptions and notation. Section B.2 presents a Cluster Preservation theorem with a formal proof, and Section B.3 provides a Generalization Bound theorem that explicitly leverages uniform convergence arguments.

B.1. Preliminaries and Assumptions

Assumption 1 (Node Embeddings.).

Each client $k$ in a federated system is associated with a local subgraph $G_{k}=(\mathcal{V}_{k},\mathcal{E}_{k})$. Let $\mathbf{h}_{k,i}^{(t)}\in\mathbb{R}^{d}$ denote the embedding of node $i\in\mathcal{V}_{k}$ at round $t$ (the dimension $d$ is typically the GNN’s output dimension). A reconstructability function $\kappa:\mathbb{R}^{d}\times\mathbb{R}^{d}\to\mathbb{R}$ maps these embeddings to a scalar:

[TABLE]

which represents the likelihood of an edge $(i,j)$ from the perspective of client $k$’s GNN. We assume $\kappa$ is $L$-Lipschitz in each argument:

Assumption 2 (Clusters and Coherence.).

We suppose there are $M$ latent clusters $C_{1},\dots,C_{M}$, each grouping clients with similar data properties, and each client $k$ is primarily associated with one of them, denoted $C(k)$. If $k$ and $m$ share the same cluster, they produce similar embeddings on a reference node set $\mathcal{V}_{\mathrm{rand}}$:

[TABLE]

for some small $\epsilon_{C}>0$. Clients in different clusters can exceed this limit, ensuring a separation in their node embeddings.

Assumption 3 (Easy vs. Hard Edges.).

Each client $k$ partitions its local edge set $\mathcal{E}_{k}$ into easy edges $\mathcal{E}_{k}^{e}$ and hard edges $\mathcal{E}_{k}^{h}$, such that at round $t$,

[TABLE]

Incremental Edge Selection* (IES) retains all easy edges from the outset and gradually incorporates a fraction of the hard edges, thus restricting local subgraph complexity early in training.*

B.2. Cluster Preservation

Let $\tilde{\mathbf{A}}_{k}^{(t)}\in\mathbb{R}^{|\tilde{\mathcal{V}}|\times|\tilde{\mathcal{V}}|}$ be the reconstructed adjacency matrix of client $k$ at round $t$, whose $(i,j)$ entry is $r_{k}^{(t)}(i,j)$, derived from the same graph among all clients. We prove that same-cluster pairs are strictly closer in Frobenius norm than cross-cluster pairs.

(Restated) Theorem 1

(Cluster Preservation) Suppose clients $k$ and $m$ belong to the same cluster $C$ of similar data properties, whereas client $n$ does not. Let $\tilde{\mathbf{A}}_{k}^{(t)}$, $\tilde{\mathbf{A}}_{m}^{(t)}$, and $\tilde{\mathbf{A}}_{n}^{(t)}$ be their reconstructed adjacency matrices at round $t$ for the same input graph. Under the Lipschitz property of $\kappa$ and the coherence assumption in Equation (7), there exists $\xi>0$ such that

[TABLE]

for some small $\epsilon_{C}>0$. Hence, same-cluster reconstructions remain closer than cross-cluster reconstructions.

Proof.

We analyze the squared Frobenius norm of the difference in reconstructability:

[TABLE]

Suppose clients $p$ and $q$ lie in the same cluster, so by (7), we have $\mathbb{E}_{i\in\mathcal{V}_{\mathrm{rand}}}\|\mathbf{h}_{p,i}^{(t)}-\mathbf{h}_{q,i}^{(t)}\|\leq\epsilon_{C}.$ Since $r_{p}^{(t)}(i,j)=\kappa(\mathbf{h}_{p,i}^{(t)},\mathbf{h}_{p,j}^{(t)})$ and $\kappa$ is $L$-Lipschitz, it follows that

[TABLE]

Averaging this bound over nodes $i,j$ in $\mathcal{V}_{\mathrm{rand}}$ yields a difference of order $O(\epsilon_{C})$. Summing across all $(i,j)$ then shows

[TABLE]

In contrast, if clients $p$ and $q$ belong to different clusters, their node embedding distributions diverge more substantially, leading to a typical difference of at least $\xi\,\epsilon_{C}$ for some $\xi>1$. Thus, the cross-cluster difference $\|\tilde{\mathbf{A}}_{p}^{(t)}-\tilde{\mathbf{A}}_{q}^{(t)}\|_{F}$ becomes $\Omega(\xi\,\epsilon_{C})$. Combining these yields the ratio implied by (8), namely that

[TABLE]

∎

Equation (8) shows that clients in the same cluster produce near-identical reconstructed adjacency matrices; this sharp contrast with cross-cluster reconstructions—revealed by the shared random graph—lets the server reliably infer clusters during training.

B.3. Generalization Bound and Overfitting Control

We prove how IES prevents overfitting by restricting the fraction of harder edges, thereby controlling the complexity of each client’s local training. Define $\mathcal{L}_{k}^{(t)}$ as the training loss of client $k$ at round $t$ (empirically measured on the retained edges) and $\mathcal{L}_{k}^{\text{test}}$ as the expected test loss under the true data distribution. Let $|\mathcal{E}_{k}^{e}|$ be the number of easy edges and $|\mathcal{E}_{k}^{h}|$ the number of hard edges in $\mathcal{E}_{k}$.

Theorem 1 (Generalization Bound).

Suppose that at round $t$, client $k$ retains all its easy edges $\mathcal{E}_{k}^{e}$ and a fraction $0\leq\lambda^{(t)}\leq 1$ of its hard edges $\mathcal{E}_{k}^{h}$, so the local subgraph used has size at most $|\mathcal{E}_{k}^{e}|+\lambda^{(t)}|\mathcal{E}_{k}^{h}|$. Then, under standard uniform-convergence arguments (e.g.,  Rademacher complexity or a VC-type bound), there exists a universal constant $c_{0}>0$ such that, with probability at least $1-\delta$,

[TABLE]

Hence, limiting the fraction of hard edges $\lambda^{(t)}$ shrinks the capacity term, reducing the gap between training loss and expected test loss.

Proof.

By uniform convergence (see, e.g., (Bartlett2002Rademacher, 3)), the generalization error $\mathcal{L}_{k}^{\text{test}}(\mathbf{w})-\mathcal{L}_{k}^{(t)}(\mathbf{w})$ can be bounded by a function of the hypothesis-class capacity. In adjacency-based GNNs, restricting the local subgraph to $|\mathcal{E}_{k}^{e}|+\lambda^{(t)}|\mathcal{E}_{k}^{h}|$ edges decreases the number of possible adjacency patterns and thus the effective size of the function class. Rademacher complexity or VC-bounds then yield

[TABLE]

Adding $\mathcal{L}_{k}^{(t)}$ to both sides of this capacity term implies (9), completing the proof. ∎

Inequality (9) states that the test loss is upper-bounded by the training loss plus a function of (i) the fraction $\lambda^{(t)}$ of hard edges and (ii) the usual $\sqrt{\log(1/\delta)/|\mathcal{E}_{k}|}$ deviation term. As $\lambda^{(t)}$ remains small in early rounds, local overfitting is greatly mitigated. Once the GNN stabilizes through federated updates, $\lambda^{(t)}$ may grow without incurring excessive variance.

Appendix C Adaptive $\tau$ Scheduler

In this section, we introduce our adaptive $\tau$ scheduler, briefly mentioned in Section 5.3.4. Similar to the GreedyLR scheduler (subramanian2023zeroth, 33), the adaptive $\tau$ scheduler greedily updates the personalized $\tau$ for each client at the beginning of specific training rounds. The main intuition here is that the optimal $\tau$ varies across clients in personalized Subgraph FL employing weighted model aggregation, as empirically demonstrated in Figure 7. More concretely, while the higher $\tau$ tends to be preferred as the data heterogeneity gets severe, the $\tau$ value that yields optimal performance for the local GNN varies from client to client (See Table 4).

The adaptive $\tau$ is outlined as follows. In the first round, the adaptive $\tau$ scheduler initializes $\tau$ to 5, and $r_{good}$ and $r_{bad}$ to 0, which denote the number of consecutive improved and deteriorated rounds, respectively. The parameter $\eta\in\{-1,1\}$, which is initialized to 1, determines the direction of the update. If the number of consecutive performance changes, whether $r_{good}$ or $r_{bad}$, surpass the $patience$ set to 5, $\tau$ is adjusted by scaling it with an update step size $\xi$, which is fixed at 1.25, depending on the sign of $\eta$. The value of $\tau$ is clipped to $\tau_{max}=10$ when it exceeds the upper bound, or to $\tau_{min}=3$ when it falls below this minimum. The update interval for $\tau$ is set to either 1 or 5.

In Algorithm 3, successive performance changes are tracked (Lines 1-7). The adaptive $\tau$ scheduler measures the client’s performance changes based on the accuracy of the aggregated local GNN. If the number of continuous performance improvements exceeds the specified $patience$, $\tau$ is adjusted in the current direction of the update (Lines 8-10). Conversely, if the number of continuous performance declines goes beyond the $patience$, the update direction $\eta$ is reversed, and $\tau$ is subsequently refined (Lines 11-14). $\tau$ is not altered if the performance variation stays within the $patience$ (Lines 15-16). In the final step, $\tau$ is constrained to lie within the prescribed range (Line 18). By greedily modifying $\tau$ based on performance trends, the adaptive $\tau$ scheduler calibrates the intensity of collaboration within the community, steering the model towards enhanced performance. Consequently, the dynamic update of $\tau$ effectively elevates the performance of personalized FL that utilizes the similarity matrix.

Appendix D Experimental Setups

D.1. Dataset Descriptions

Table 5 presents statistics of the six datasets used in our experiments. These include small citation graphs such as Cora, CiteSeer, and PubMed (DBLP:conf/icml/YangCS16, 44); product graphs such as Amazon-Computer and Amazon-Photo (DBLP:journals/corr/abs-1811-05868, 32); and a large citation graph, ogbn-arxiv (DBLP:conf/nips/HuFZDRLCL20, 12). In our main experimental setting, the local subgraphs are the output of METIS (karypis1997metis, 14), ensuring no overlapping nodes between them. Due to the missing links between local subgraphs, the total number of edges across clients is lower than the number of edges in the full graph. Furthermore, the homophily principle (mcpherson2001birds, 27), which states that nodes with the same label are more likely to be connected, leads to heterogeneity in class distribution (See Figure 8).

D.2. Baselines

D.2.1. FedAvgCL

FedAvgCL is a centralized Subgraph FL baseline developed in this work, which applies Incremental Edge Selection (IES) to FedAvg (DBLP:conf/aistats/McMahanMRHA17, 26). During the local training stage, local GNN is optimized following the curriculum provided by IES. In the aggregation stage, the server aggregates the local GNNs with respect to the number of nodes in each local subgraph. FedAvgCL demonstrates the necessity of adopting Curriculum Learning (CL) in Subgraph FL and the importance of weighted model aggregation in CUFL.

D.2.2. FedGNN

FedGNN (DBLP:journals/corr/abs-2102-04925, 42) augments local subgraphs using overlapping nodes as expansion pivots. This design, however, is not applicable to our experimental setting, where client subgraphs are strictly disjoint. For fair comparison, we adapt FedGNN to the node non-overlapping scenario. Concretely, we sample 10% of nodes from each client to construct a shared candidate pool, which serves as a surrogate for overlaps. Each local node subsequently augments its neighborhood with candidates chosen based on cosine similarity in the embedding space of the global model. The number of attached neighbors for each node is 20 for Cora and CiteSeer, 10 for PubMed, Amazon-Computer, and Amazon-Photo, and 1 for ogbn-arxiv.

D.3. Additional Ablation Studies

D.3.1. Results on Louvain Partitioning

To examine the robustness of CUFL under alternative subgraph partitioning schemes, we apply CUFL to subgraphs generated by the Louvain algorithm (DBLP:journals/corr/abs-2311-06047, 5). The Louvain method greedily maximizes modularity in community detection, thereby favoring structurally homogeneous clusters. As a result, structurally similar clients become more pronounced, but the resulting subgraph sizes are highly imbalanced. Since the Louvain method inherently leaves the number of partitions unspecified, we adopt the strategy of (DBLP:conf/nips/ZhangYLSY21, 47) to randomly merge the resulting subgraphs irrespective of graph properties. Table 6 demonstrates that CUFL achieves strong performance on Louvain-partitioned subgraphs. The variance across runs remains limited, further suggesting that CUFL exhibits stable behavior under this partitioning.

D.3.2. Influence of Random Graph Configurations

The random graphs allow CUFL to estimate client similarity without exposing private data, yet our experiments rely solely on the Stochastic Block Model (SBM) with a fixed graph size. In this section, we examine how random graph selection affects performance. Accordingly, we vary the random graph model and size.

Regarding the random graph model, we substitute SBM with two widely studied alternatives—Erdős–Rényi (ER) and Barabási–Albert (BA) (DBLP:journals/corr/abs-2403-14415, 6), while keeping the number of nodes consistent with the hyperparameter setting. ER draws each edge independently with a fixed probability, producing a nearly binomial degree distribution and virtually no intrinsic community structure, whereas BA grows the network by preferential attachment, yielding a heavy-tailed degree distribution dominated by hub nodes. Table 7 summarizes the performance on Cora and PubMed, showing that both SBM and BA consistently outperform ER. SBM and BA encode salient citation-network patterns—communities and hub-dominated connectivity—whereas ER remains structure-agnostic. This indicates that CUFL performs better when the random graphs encode the clients’ structural statistics.

Orthogonal to the choice of random graph model, we examine the impact of graph size by varying the number of nodes. All other settings are kept consistent with the original hyperparameters. As shown in Table 8, undersized random graphs fail to capture representative features of local GNNs. Increasing the size provides more structural information, which initially improves performance. However, once the size reaches a large scale, improvements become marginal and, in some cases, larger graphs impair accuracy. While larger random graphs can improve performance, their size simultaneously increases the complexity of the server aggregation stage. Therefore, before the random graph size becomes excessively large, a trade-off emerges between performance and complexity.