# FedDyH: A Multi-Policy with GA Optimization Framework for Dynamic Heterogeneous Federated Learning

## TL;DR

FedDyH is a new federated learning framework that uses biological-inspired methods to handle dynamic data changes and improve model accuracy in distributed settings.

## Contribution

FedDyH introduces a novel framework combining cross-client knowledge distillation, dynamic regularization, and genetic algorithm optimization to address dynamic heterogeneity in federated learning.

## Key findings

- FedDyH outperforms the SOTA FedDecorr baseline by 2.59%, 0.55%, and 5.79% on MNIST, Fashion-MNIST, and CIFAR-10 datasets.
- The framework effectively handles dynamic heterogeneity through adaptive regularization and genetic algorithm-based hyperparameter optimization.
- The proposed methods improve model robustness and accuracy in changing environments.

## Abstract

Federated learning (FL) is a distributed learning technique that ensures data privacy and has shown significant potential in cross-institutional image analysis. However, existing methods struggle with the inherent dynamic heterogeneity of real-world data, such as changes in cellular differentiation during disease progression or feature distribution shifts due to different imaging devices. This dynamic heterogeneity can cause catastrophic forgetting, leading to reduced performance in medical predictions across stages. Unlike previous federated learning studies that paid insufficient attention to dynamic heterogeneity, this paper proposes the FedDyH framework to address this challenge. Inspired by the adaptive regulation mechanisms of biological systems, this framework incorporates several core modules to tackle the issues arising from dynamic heterogeneity. First, the framework simulates intercellular information transfer through cross-client knowledge distillation, preserving local features while mitigating knowledge forgetting. Additionally, a dynamic regularization term is designed in which the strength can be adaptively adjusted based on real-world conditions. This mechanism resembles the role of regulatory T cells in the immune system, balancing global model convergence with local specificity adjustments to enhance the robustness of the global model while preventing interference from diverse client features. Finally, the framework introduces a genetic algorithm (GA) to simulate biological evolution, leveraging mechanisms such as gene selection, crossover, and mutation to optimize hyperparameter configurations. This enables the model to adaptively find the optimal hyperparameters in an ever-changing environment, thereby improving both adaptability and performance. Prior to this work, few studies have explored the use of optimization algorithms for hyperparameter tuning in federated learning. Experimental results demonstrate that the FedDyH framework improves accuracy compared to the SOTA baseline FedDecorr by 2.59%, 0.55%, and 5.79% on the MNIST, Fashion-MNIST, and CIFAR-10 benchmark datasets, respectively. This framework effectively addresses data heterogeneity issues in dynamic heterogeneous environments, providing an innovative solution for achieving more stable and accurate distributed federated learning.