FedPID: An Aggregation Method for Federated Learning

TL;DR

FedPID introduces an improved aggregation strategy for federated learning inspired by PID control, addressing dataset size disparities and enhancing model convergence in collaborative training.

Contribution

The paper proposes FedPID, an enhanced aggregation method incorporating PID-inspired control and dataset size adjustments, advancing federated learning performance.

Findings

Effective handling of varying dataset sizes at centers.

Improved convergence in federated learning tasks.

Successful application in FETS24 challenge.

Abstract

This paper presents FedPID, our submission to the Federated Tumor Segmentation Challenge 2024 (FETS24). Inspired by FedCostWAvg and FedPIDAvg, our winning contributions to FETS21 and FETS2022, we propose an improved aggregation strategy for federated and collaborative learning. FedCostWAvg is a method that averages results by considering both the number of training samples in each group and how much the cost function decreased in the last round of training. This is similar to how the derivative part of a PID controller works. In FedPIDAvg, we also included the integral part that was missing. Another challenge we faced were vastly differing dataset sizes at each center. We solved this by assuming the sizes follow a Poisson distribution and adjusting the training iterations for each center accordingly. Essentially, this part of the method controls that outliers that require too much training time are less frequently used. Based on these contributions we now adapted FedPIDAvg by changing how the integral part is computed. Instead of integrating the loss function we measure the global drop in cost since the first round.