ECAvg: An Edge-Cloud Collaborative Learning Approach using Averaged Weights

TL;DR

ECAvg is a collaborative edge-cloud learning method where edge devices pre-train models, which are then averaged and fine-tuned on the server, improving performance on complex datasets but not on simpler ones.

Contribution

The paper introduces ECAvg, a novel edge-cloud collaborative learning approach using weight averaging and fine-tuning, effective for deep neural networks on complex datasets.

Findings

Improved performance on CIFAR-10 and CIFAR-100 with ECAvg.

Performance drop on MNIST due to negative transfer.

Effective for deep neural networks like MobileNetV2 and ResNet50.

Abstract

The use of edge devices together with cloud provides a collaborative relationship between both classes of devices where one complements the shortcomings of the other. Resource-constraint edge devices can benefit from the abundant computing power provided by servers by offloading computationally intensive tasks to the server. Meanwhile, edge devices can leverage their close proximity to the data source to perform less computationally intensive tasks on the data. In this paper, we propose a collaborative edge-cloud paradigm called ECAvg in which edge devices pre-train local models on their respective datasets and transfer the models to the server for fine-tuning. The server averages the pre-trained weights into a global model, which is fine-tuned on the combined data from the various edge devices. The local (edge) models are then updated with the weights of the global (server) model. We implement a CIFAR-10 classification task using MobileNetV2, a CIFAR-100 classification task using ResNet50, and an MNIST classification using a neural network with a single hidden layer. We observed performance improvement in the CIFAR-10 and CIFAR-100 classification tasks using our approach, where performance improved on the server model with averaged weights and the edge models had a better performance after model update. On the MNIST classification, averaging weights resulted in a drop in performance on both the server and edge models due to negative transfer learning. From the experiment results, we conclude that our approach is successful when implemented on deep neural networks such as MobileNetV2 and ResNet50 instead of simple neural networks.