CA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction

TL;DR

CA-HFP introduces a curvature-aware, personalized pruning framework for federated learning on heterogeneous devices, enabling efficient model compression and reconstruction while maintaining accuracy and convergence.

Contribution

The paper proposes a novel curvature-informed, device-specific pruning method with a lightweight model reconstruction in federated learning, addressing heterogeneity and efficiency.

Findings

Achieves significant reduction in computation and communication costs.

Maintains high model accuracy across diverse datasets and architectures.

Outperforms existing federated training and pruning baselines.

Abstract

Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing per-client computation and communication costs, outperforming standard federated training and existing pruning-based baselines.