FedAvgen: Metadata for Model Aggregation In Communication Systems

TL;DR

FedAvgen introduces a genetic algorithm-based method for model aggregation in federated learning, addressing device diversity challenges and improving upon standard algorithms like FedAvg and FedSGD.

Contribution

The paper proposes a novel metaheuristic algorithm, FedAvgen, that models model aggregation as a genetic evolution process using metadata, enhancing federated learning in communication systems.

Findings

FedAvgen outperforms FedAvg and FedSGD in diverse device scenarios.

The genetic approach improves global model accuracy and robustness.

Metaheuristic-based aggregation offers better adaptation to device heterogeneity.

Abstract

To improve business efficiency and minimize costs, Artificial Intelligence (AI) practitioners have adopted a shift from formulating models from scratch towards sharing pretrained models. The pretrained models are then aggregated into a global model with higher generalization capabilities, which is afterwards distributed to the client devices. This approach is known as federated learning and inherently utilizes different techniques to select the candidate client models averaged to obtain the global model. This approach, in the case of communication systems, faces challenges arising from the existential diversity in device profiles. The multiplicity in profiles motivates our conceptual assessment of a metaheuristic algorithm (FedAvgen), which relates each pretrained model with its weight space as metadata, to a phenotype and genotype, respectively. This parent-child genetic evolution characterizes the global averaging step in federated learning. We then compare the results of our approach to two widely adopted baseline federated learning algorithms like Federated Averaging (FedAvg) and Federated Stochastic Gradient Descent (FedSGD).