MAcPNN: Mutual Assisted Learning on Data Streams with Temporal Dependence

TL;DR

This paper introduces MAcPNN, a decentralized mutual learning framework for IoT data streams that enhances model performance by enabling autonomous devices to collaborate selectively, reducing communication overhead and handling concept drifts effectively.

Contribution

It proposes a novel mutual assisted learning paradigm based on sociocultural theory, utilizing cPNNs with quantization for efficient, decentralized, and adaptive IoT data stream analysis.

Findings

Improves performance on synthetic and real data streams.

Reduces communication overhead compared to federated learning.

Effectively handles concept drifts in IoT environments.

Abstract

Internet of Things (IoT) Analytics often involves applying machine learning (ML) models on data streams. In such scenarios, traditional ML paradigms face obstacles related to continuous learning while dealing with concept drifts, temporal dependence, and avoiding forgetting. Moreover, in IoT, different edge devices build up a network. When learning models on those devices, connecting them could be useful in improving performance and reusing others' knowledge. This work proposes Mutual Assisted Learning, a learning paradigm grounded on Vygotsky's popular Sociocultural Theory of Cognitive Development. Each device is autonomous and does not need a central orchestrator. Whenever it degrades its performance due to a concept drift, it asks for assistance from others and decides whether their knowledge is useful for solving the new problem. This way, the number of connections is drastically reduced compared to the classical Federated Learning approaches, where the devices communicate at each training round. Every device is equipped with a Continuous Progressive Neural Network (cPNN) to handle the dynamic nature of data streams. We call this implementation Mutual Assisted cPNN (MAcPNN). To implement it, we allow cPNNs for single data point predictions and apply quantization to reduce the memory footprint. Experimental results prove the effectiveness of MAcPNN in boosting performance on synthetic and real data streams.