Revisiting IoT Device Identification

TL;DR

This paper evaluates machine learning models for IoT device identification based on network behavior, revealing that model accuracy declines over time and emphasizing the need for continuous updates.

Contribution

It compares the accuracy of four machine learning models for IoT device identification and highlights the importance of model updating to maintain high accuracy.

Findings

Models achieve high initial accuracy but degrade over weeks.

Accuracy drops by up to 40 percentage points over time.

Continuous model updating is necessary for reliable device identification.

Abstract

Internet-of-Things (IoT) devices are known to be the source of many security problems, and as such, they would greatly benefit from automated management. This requires robustly identifying devices so that appropriate network security policies can be applied. We address this challenge by exploring how to accurately identify IoT devices based on their network behavior, while leveraging approaches previously proposed by other researchers. We compare the accuracy of four different previously proposed machine learning models (tree-based and neural network-based) for identifying IoT devices. We use packet trace data collected over a period of six months from a large IoT test-bed. We show that, while all models achieve high accuracy when evaluated on the same dataset as they were trained on, their accuracy degrades over time, when evaluated on data collected outside the training set. We show that on average the models' accuracy degrades after a couple of weeks by up to 40 percentage points (on average between 12 and 21 percentage points). We argue that, in order to keep the models' accuracy at a high level, these need to be continuously updated.