Applied Federated Model Personalisation in the Industrial Domain: A Comparative Study

TL;DR

This paper compares three personalization strategies—Active Learning, Knowledge Distillation, and Local Memorization—in federated learning for industrial IoT, demonstrating their effectiveness in improving model accuracy and efficiency.

Contribution

It introduces an advanced federated learning system that applies these personalization methods, providing a comparative analysis of their impact on model performance in industrial IoT applications.

Findings

Optimized models show improved accuracy in federated settings.

Personalization techniques reduce computational resources needed.

Encouraging results in real-time IoT applications.

Abstract

The time-consuming nature of training and deploying complicated Machine and Deep Learning (DL) models for a variety of applications continues to pose significant challenges in the field of Machine Learning (ML). These challenges are particularly pronounced in the federated domain, where optimizing models for individual nodes poses significant difficulty. Many methods have been developed to tackle this problem, aiming to reduce training expenses and time while maintaining efficient optimisation. Three suggested strategies to tackle this challenge include Active Learning, Knowledge Distillation, and Local Memorization. These methods enable the adoption of smaller models that require fewer computational resources and allow for model personalization with local insights, thereby improving the effectiveness of current models. The present study delves into the fundamental principles of these three approaches and proposes an advanced Federated Learning System that utilises different Personalisation methods towards improving the accuracy of AI models and enhancing user experience in real-time NG-IoT applications, investigating the efficacy of these techniques in the local and federated domain. The results of the original and optimised models are then compared in both local and federated contexts using a comparison analysis. The post-analysis shows encouraging outcomes when it comes to optimising and personalising the models with the suggested techniques.