An Experiment Study on Federated LearningTestbed

TL;DR

This paper evaluates the efficiency and privacy aspects of a federated learning testbed for IoT, highlighting communication delays and security vulnerabilities, and providing insights for future PPML framework improvements.

Contribution

It offers an empirical analysis of a PySyft-based federated learning framework's performance and security, guiding future development in privacy-preserving IoT machine learning.

Findings

Training speed is significantly slower than centralized methods due to communication overhead.

The framework has vulnerabilities to man-in-the-middle attacks.

Provides a baseline for future PPML performance and security improvements.

Abstract

While the Internet of Things (IoT) can benefit from machine learning by outsourcing model training on the cloud, user data exposure to an untrusted cloud service provider can pose threat to user privacy. Recently, federated learning is proposed as an approach for privacy-preserving machine learning (PPML) for the IoT, while its practicability remains unclear. This work presents the evaluation on the efficiency and privacy performance of a readily available federated learning framework based on PySyft, a Python library for distributed deep learning. It is observed that the training speed of the framework is significantly slower than of the centralized approach due to communication overhead. Meanwhile, the framework bears some vulnerability to potential man-in-the-middle attacks at the network level. The report serves as a starting point for PPML performance analysis and suggests the future direction for PPML framework development.