Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy

TL;DR

This paper introduces a novel data-free method using spectral entropy of client updates to estimate contributions in federated learning, avoiding privacy issues and manipulation risks.

Contribution

The authors propose SpectralFed and SpectralFuse, two schemes leveraging von Neumann entropy for client contribution estimation without validation data.

Findings

Spectral entropy correlates strongly with client accuracy across multiple benchmarks.

The methods perform well under diverse non-IID data distributions.

Spectral entropy outperforms existing data-free contribution estimation baselines.

Abstract

Client contribution estimation in Federated Learning is necessary for identifying clients' importance and for providing fair rewards. Current methods often rely on server-side validation data or self-reported client information, which can compromise privacy or be susceptible to manipulation. We introduce a data-free signal based on the matrix von Neumann (spectral) entropy of the final-layer updates, which measures the diversity of the information contributed. We instantiate two practical schemes: (i) SpectralFed, which uses normalized entropy as aggregation weights, and (ii) SpectralFuse, which fuses entropy with class-specific alignment via a rank-adaptive Kalman filter for per-round stability. Across CIFAR-10/100 and the naturally partitioned FEMNIST and FedISIC benchmarks, entropy-derived scores show a consistently high correlation with standalone client accuracy under diverse non-IID regimes - without validation data or client metadata. We compare our results with data-free contribution estimation baselines and show that spectral entropy serves as a useful indicator of client contribution.