FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling

TL;DR

FedOBP introduces a novel federated personalization method that uses element-wise importance scores based on saliency theory, optimizing personalized parameters while reducing client resource demands.

Contribution

This work bridges saliency-based pruning theory with federated parameter decoupling, providing a principled approach for selecting personalized model parameters.

Findings

FedOBP outperforms existing methods on multiple datasets.

Requires only a small subset of parameters to be personalized.

Alleviates client resource burden by moving metric computation to server.

Abstract

Federated Learning (FL) faces challenges from client data heterogeneity and resource-constrained mobile devices, which can degrade model accuracy. Personalized Federated Learning (PFL) addresses this issue by adapting shared global knowledge to local data distributions. A promising approach in PFL is model decoupling, which separates the model into global and personalized parameters, raising the key question of which parameters should be personalized to balance global knowledge sharing and local adaptation. In this paper, we propose a Federated Optimal Brain Personalization (FedOBP) algorithm with a quantile-based thresholding mechanism and introduce an element-wise importance score. This score extends Optimal Brain Damage (OBD) pruning theory by incorporating a federated approximation of the first-order derivative in the Taylor expansion to evaluate the importance of each parameter for personalization. Moreover, we move the metric computation originally performed on clients to the server side, to alleviate the burden on resource-constrained mobile devices. To the best of our knowledge, this is the first work to bridge classical saliency-based pruning theory with federated parameter decoupling, providing a rigorous theoretical justification for selecting personalized parameters based on their sensitivity to local loss landscapes. Extensive experiments demonstrate that FedOBP outperforms state-of-the-art methods across diverse datasets and heterogeneity scenarios, while requiring personalization of only a very small number of personalized parameters.