Fisher-Informed Parameterwise Aggregation for Federated Learning with Heterogeneous Data

TL;DR

FIPA introduces a second-order, parameter-specific aggregation method for federated learning that adapts to data heterogeneity, improving model performance across various tasks.

Contribution

The paper proposes Fisher-Informed Parameterwise Aggregation (FIPA), a novel second-order method that uses Fisher Information Matrix weights for parameter-level client update aggregation.

Findings

FIPA outperforms standard averaging in diverse tasks.

FIPA remains communication-efficient with low-rank approximation.

FIPA enhances accuracy when combined with advanced client optimizers.

Abstract

Federated learning aggregates model updates from distributed clients, but standard first order methods such as FedAvg apply the same scalar weight to all parameters from each client. Under non-IID data, these uniformly weighted updates can be strongly misaligned across clients, causing client drift and degrading the global model. Here we propose Fisher-Informed Parameterwise Aggregation (FIPA), a second-order aggregation method that replaces client-level scalar weights with parameter-specific Fisher Information Matrix (FIM) weights, enabling true parameter-level scaling that captures how each client's data uniquely influences different parameters. With low-rank approximation, FIPA remains communication- and computation-efficient. Across nonlinear function regression, PDE learning, and image classification, FIPA consistently improves over averaging-based aggregation, and can be effectively combined with state-of-the-art client-side optimization algorithms to further improve image classification accuracy. These results highlight the benefits of FIPA for federated learning under heterogeneous data distributions.