FLsim: A Modular and Library-Agnostic Simulation Framework for Federated Learning

TL;DR

FLsim is a flexible, modular simulation framework for federated learning that supports diverse experimental setups, making it easier to benchmark and develop new FL techniques.

Contribution

Introduces FLsim, a novel, library-agnostic simulation framework that enhances reproducibility, customization, and scalability in federated learning research.

Findings

Demonstrates FLsim's effectiveness across various FL experiments.

Shows FLsim's ability to simulate diverse data distributions and network topologies.

Validates FLsim's resource efficiency and reproducibility.

Abstract

Federated Learning (FL) has undergone significant development since its inception in 2016, advancing from basic algorithms to complex methodologies tailored to address diverse challenges and use cases. However, research and benchmarking of novel FL techniques against a plethora of established state-of-the-art solutions remain challenging. To streamline this process, we introduce FLsim, a comprehensive FL simulation framework designed to meet the diverse requirements of FL workflows in the literature. FLsim is characterized by its modularity, scalability, resource efficiency, and controlled reproducibility of experimental outcomes. Its easy to use interface allows users to specify customized FL requirements through job configuration, which supports: (a) customized data distributions, ranging from non-independent and identically distributed (non-iid) data to independent and identically distributed (iid) data, (b) selection of local learning algorithms according to user preferences, with complete agnosticism to ML libraries, (c) choice of network topology illustrating communication patterns among nodes, (d) definition of model aggregation and consensus algorithms, and (e) pluggable blockchain support for enhanced robustness. Through a series of experimental evaluations, we demonstrate the effectiveness and versatility of FLsim in simulating a diverse range of state-of-the-art FL experiments. We envisage that FLsim would mark a significant advancement in FL simulation frameworks, offering unprecedented flexibility and functionality for researchers and practitioners alike.