STEAM++ An Extensible End-To-End Framework for Developing IoT Data Processing Applications in the Fog

TL;DR

STEAM++ is a lightweight, extensible framework designed for real-time IoT data processing at the network edge, especially suitable for industrial environments with limited connectivity and hardware constraints.

Contribution

It introduces STEAM++, a novel framework for efficient, real-time IoT data stream processing on resource-limited devices, along with a micro-benchmark methodology for embedded applications.

Findings

Processed data from sensing to analysis in industrial settings.

Achieved low resource consumption with less than 500KB RAM and 1% CPU.

Reduced data output size to 14% of raw data during event notification.

Abstract

IoT applications usually rely on cloud computing services to perform data analysis such as filtering, aggregation, classification, pattern detection, and prediction. When applied to specific domains, the IoT needs to deal with unique constraints. Besides the hostile environment such as vibration and electric-magnetic interference, resulting in malfunction, noise, and data loss, industrial plants often have Internet access restricted or unavailable, forcing us to design stand-alone fog and edge computing solutions. In this context, we present STEAM++, a lightweight and extensible framework for real-time data stream processing and decision-making in the network edge, targeting hardware-limited devices, besides proposing a micro-benchmark methodology for assessing embedded IoT applications. In real-case experiments in a semiconductor industry, we processed an entire data flow, from values sensing, processing and analyzing data, detecting relevant events, and finally, publishing results to a dashboard. On average, the application consumed less than 500kb RAM and 1.0% of CPU usage, processing up to 239 data packets per second and reducing the output data size to 14% of the input raw data size when notifying events.