A Framework for Simulating Real-world Stream Data of the Internet of Things

TL;DR

This paper introduces a novel framework for simulating real-world IoT stream data, enabling more efficient stream processing by accurately capturing data volatility and trends, and demonstrating significant acceleration in processing tasks.

Contribution

The paper presents a new framework for simulating IoT stream data that accurately models real-world characteristics, improving processing efficiency and scalability.

Findings

Stream processing tasks are accelerated by at least 24 times.

The framework effectively captures volatility and trends of IoT stream data.

Experimental validation confirms the framework's efficiency and accuracy.

Abstract

With the rapid growth in the number of devices of the Internet of Things (IoT), the volume and types of stream data are rapidly increasing in the real world. Unfortunately, the stream data has the characteristics of infinite and periodic volatility in the real world, which cause problems with the inefficient stream processing tasks. In this study, we report our recent efforts on this issue, with a focus on simulating stream data. Firstly, we explore the characteristics of the real-world stream data of the IoT, which helps us to understand the stream data in the real world. Secondly, the pipeline of simulating stream data is proposed, which can accurately and efficiently simulate the characteristics of the stream data to improve efficiency for specific tasks. Finally, we design and implement a novel framework that can simulate various stream data for related stream processing tasks. To verify the validity of the proposed framework, we apply this framework to stream processing task running in the stream processing system. The experimental results reveal that the related stream processing task is accelerated by at least 24 times using our proposed simulation framework with the premise of ensuring volatility and trends of stream data.