Intrinsic Training Signals for Federated Learning Aggregation

TL;DR

This paper introduces LIVAR, a novel federated learning aggregation method that leverages intrinsic training signals like feature statistics and explainability analysis, achieving state-of-the-art results without architectural modifications.

Contribution

LIVAR is the first approach to use existing training signals for federated model merging, eliminating the need for architectural or loss function changes.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Operates without architectural overhead or modifications.

Demonstrates effective model merging using only training signals.

Abstract

Federated Learning (FL) enables collaborative model training across distributed clients while preserving data privacy. While existing approaches for aggregating client-specific classification heads and adapted backbone parameters require architectural modifications or loss function changes, our method uniquely leverages intrinsic training signals already available during standard optimization. We present LIVAR (Layer Importance and VARiance-based merging), which introduces: i) a variance-weighted classifier aggregation scheme using naturally emergent feature statistics, and ii) an explainability-driven LoRA merging technique based on SHAP analysis of existing update parameter patterns. Without any architectural overhead, LIVAR achieves state-of-the-art performance on multiple benchmarks while maintaining seamless integration with existing FL methods. This work demonstrates that effective model merging can be achieved solely through existing training signals, establishing a new paradigm for efficient federated model aggregation. The code is available at https://github.com/aimagelab/fed-mammoth.