FedConv: A Learning-on-Model Paradigm for Heterogeneous Federated Clients

TL;DR

FedConv introduces a federated learning framework that creates personalized, resource-efficient sub-models for clients with diverse capabilities, using convolutional compression and dilation to improve efficiency and accuracy.

Contribution

The paper proposes FedConv, a novel learning-on-model paradigm that enables heterogeneous sub-model training and aggregation in federated learning, reducing resource demands for clients.

Findings

FedConv achieves over 35% higher accuracy than state-of-the-art methods.

Reduces computation and communication overhead by 33% and 25%.

Effectively personalizes models for resource-constrained clients.

Abstract

Federated Learning (FL) facilitates collaborative training of a shared global model without exposing clients' private data. In practical FL systems, clients (e.g., edge servers, smartphones, and wearables) typically have disparate system resources. Conventional FL, however, adopts a one-size-fits-all solution, where a homogeneous large global model is transmitted to and trained on each client, resulting in an overwhelming workload for less capable clients and starvation for other clients. To address this issue, we propose FedConv, a client-friendly FL framework, which minimizes the computation and memory burden on resource-constrained clients by providing heterogeneous customized sub-models. FedConv features a novel learning-on-model paradigm that learns the parameters of the heterogeneous sub-models via convolutional compression. Unlike traditional compression methods, the compressed models in FedConv can be directly trained on clients without decompression. To aggregate the heterogeneous sub-models, we propose transposed convolutional dilation to convert them back to large models with a unified size while retaining personalized information from clients. The compression and dilation processes, transparent to clients, are optimized on the server leveraging a small public dataset. Extensive experiments on six datasets demonstrate that FedConv outperforms state-of-the-art FL systems in terms of model accuracy (by more than 35% on average), computation and communication overhead (with 33% and 25% reduction, respectively).