Combining Individual and Joint Networking Behavior for Intelligent IoT Analytics

TL;DR

This paper introduces IoTelligent, a scalable machine learning framework for managing large-scale IoT deployments by analyzing network traffic data, forecasting demand, and detecting anomalous devices using CNN, LSTM, and autoencoders.

Contribution

The work presents a novel scalable approach combining demand forecasting and device management techniques tailored for industrial IoT environments.

Findings

Convolutional LSTM outperforms other models in demand forecasting.

A general demand model with normalization effectively manages multiple companies.

Autoencoders successfully identify device groups and anomalies.

Abstract

The IoT vision of a trillion connected devices over the next decade requires reliable end-to-end connectivity and automated device management platforms. While we have seen successful efforts for maintaining small IoT testbeds, there are multiple challenges for the efficient management of large-scale device deployments. With Industrial IoT, incorporating millions of devices, traditional management methods do not scale well. In this work, we address these challenges by designing a set of novel machine learning techniques, which form a foundation of a new tool, it IoTelligent, for IoT device management, using traffic characteristics obtained at the network level. The design of our tool is driven by the analysis of 1-year long networking data, collected from 350 companies with IoT deployments. The exploratory analysis of this data reveals that IoT environments follow the famous Pareto principle, such as: (i) 10% of the companies in the dataset contribute to 90% of the entire traffic; (ii) 7% of all the companies in the set own 90% of all the devices. We designed and evaluated CNN, LSTM, and Convolutional LSTM models for demand forecasting, with a conclusion of the Convolutional LSTM model being the best. However, maintaining and updating individual company models is expensive. In this work, we design a novel, scalable approach, where a general demand forecasting model is built using the combined data of all the companies with a normalization factor. Moreover, we introduce a novel technique for device management, based on autoencoders. They automatically extract relevant device features to identify device groups with similar behavior to flag anomalous devices.