Federated Learning of Neural ODE Models with Different Iteration Counts

TL;DR

This paper introduces a federated learning method using Neural ODE models that reduces communication costs and supports heterogeneous client models with different iteration counts, demonstrating significant efficiency gains.

Contribution

The paper presents a novel federated learning approach with Neural ODE models that effectively aggregates models of varying depths and reduces communication overhead.

Findings

Achieves up to 92.4% reduction in communication size.

Successfully aggregates models with different iteration counts.

Maintains comparable accuracy to existing federated learning methods.

Abstract

Federated learning is a distributed machine learning approach in which clients train models locally with their own data and upload them to a server so that their trained results are shared between them without uploading raw data to the server. There are some challenges in federated learning, such as communication size reduction and client heterogeneity. The former can mitigate the communication overheads, and the latter can allow the clients to choose proper models depending on their available compute resources. To address these challenges, in this paper, we utilize Neural ODE based models for federated learning. The proposed flexible federated learning approach can reduce the communication size while aggregating models with different iteration counts or depths. Our contribution is that we experimentally demonstrate that the proposed federated learning can aggregate models with different iteration counts or depths. It is compared with a different federated learning approach in terms of the accuracy. Furthermore, we show that our approach can reduce communication size by up to 92.4% compared with a baseline ResNet model using CIFAR-10 dataset.