On-device edge learning for IoT data streams: a survey

TL;DR

This survey reviews continual learning techniques for on-device neural networks and decision trees in IoT, emphasizing challenges like resource constraints, data stream handling, and model adaptation for edge environments.

Contribution

It provides a comprehensive overview of current methods, challenges, and evaluation criteria for deploying continual learning models on resource-limited edge devices in IoT.

Findings

Decision trees are more memory-efficient but less expressive.

Handling concept drift requires dynamic model adaptations.

Multi-criteria evaluation is crucial for edge applications.

Abstract

This literature review explores continual learning methods for on-device training in the context of neural networks (NNs) and decision trees (DTs) for classification tasks on smart environments. We highlight key constraints, such as data architecture (batch vs. stream) and network capacity (cloud vs. edge), which impact TinyML algorithm design, due to the uncontrolled natural arrival of data streams. The survey details the challenges of deploying deep learners on resource-constrained edge devices, including catastrophic forgetting, data inefficiency, and the difficulty of handling IoT tabular data in open-world settings. While decision trees are more memory-efficient for on-device training, they are limited in expressiveness, requiring dynamic adaptations, like pruning and meta-learning, to handle complex patterns and concept drifts. We emphasize the importance of multi-criteria performance evaluation tailored to edge applications, which assess both output-based and internal representation metrics. The key challenge lies in integrating these building blocks into autonomous online systems, taking into account stability-plasticity trade-offs, forward-backward transfer, and model convergence.