Leveraging Learning Metrics for Improved Federated Learning

TL;DR

This paper introduces a novel federated learning metric called Effective Rank, derived from XAI research, and demonstrates its effectiveness in improving model aggregation and performance over traditional methods.

Contribution

It presents the first federated learning metric aggregation method based on Effective Rank and develops a new weight-aggregation scheme leveraging this metric.

Findings

Effective Rank outperforms Federated Averaging in experiments

The proposed aggregation scheme improves model convergence

Effective Rank effectively measures layer mapping quality in federated settings

Abstract

Currently in the federated setting, no learning schemes leverage the emerging research of explainable artificial intelligence (XAI) in particular the novel learning metrics that help determine how well a model is learning. One of these novel learning metrics is termed `Effective Rank' (ER) which measures the Shannon Entropy of the singular values of a matrix, thus enabling a metric determining how well a layer is mapping. By joining federated learning and the learning metric, effective rank, this work will \textbf{(1)} give the first federated learning metric aggregation method \textbf{(2)} show that effective rank is well-suited to federated problems by out-performing baseline Federated Averaging \cite{konevcny2016federated} and \textbf{(3)} develop a novel weight-aggregation scheme relying on effective rank.