# Systematic Evaluation of Machine Learning and Deep Learning Models for IoT Malware Detection Across Ransomware, Rootkit, Spyware, Trojan, Botnet, Worm, Virus, and Keylogger

## TL;DR

This study compares machine learning and deep learning models for detecting IoT malware, finding that tree-based models like CatBoost and LightGBM perform best overall and are more efficient for most malware types.

## Contribution

The paper systematically evaluates 45 ML/DL models across eight IoT malware categories using a large, realistic dataset, revealing malware-specific model preferences and efficiency advantages of tree-based models.

## Key findings

- Ensemble tree-based models outperform deep learning models for most IoT malware detection tasks.
- CatBoost and LightGBM achieve near-ceiling accuracy with low false-positive rates across multiple malware categories.
- Deep learning models are only competitive in specific cases like ransomware and virus detection.

## Abstract

What are the main findings?
Ensemble tree-based machine learning models outperform deep learning architectures for feature-engineered IoT malware telemetry, with 7 of the top 10 models being ML-based.Optimal detection performance is malware-dependent, with gradient-boosted trees dominating most categories and tabular deep learning models excelling only in specific cases such as ransomware and virus detection.

Ensemble tree-based machine learning models outperform deep learning architectures for feature-engineered IoT malware telemetry, with 7 of the top 10 models being ML-based.

Optimal detection performance is malware-dependent, with gradient-boosted trees dominating most categories and tabular deep learning models excelling only in specific cases such as ransomware and virus detection.

What are the implications of the main findings?
Gradient-boosted ensemble ML models provide the best accuracy–efficiency trade-off for practical IoT malware detection on commodity hardware.Model selection for IoT security should be data- and threat-aware, favoring classical ML for engineered telemetry and reserving deep learning for behavior-specific scenarios.

Gradient-boosted ensemble ML models provide the best accuracy–efficiency trade-off for practical IoT malware detection on commodity hardware.

Model selection for IoT security should be data- and threat-aware, favoring classical ML for engineered telemetry and reserving deep learning for behavior-specific scenarios.

The rapid growth of Internet-of-Things (IoT) deployments has substantially expanded the attack surface of modern cyber–physical systems, making accurate and computationally feasible malware detection essential for enterprise and industrial environments. This study presents a large-scale, systematic comparison of 27 machine learning (ML) and 18 deep learning (DL) models for IoT malware detection across eight major malware categories: Trojan, Botnet, Ransomware, Rootkit, Worm, Spyware, Keylogger, and Virus. A realistic dataset was constructed using 50,000 executable samples collected from the Any.Run platform, including 8000 malware instances (1000 per class) and 42,000 benign samples. Each sample was executed in a sandbox to extract detailed static and behavioral telemetry. A targeted feature-selection pipeline reduced the feature space to 47 diagnostic features spanning static properties, behavioral indicators, process/file/registry activity, debug signals, and network telemetry, yielding a compact representation suitable for malware detection in IoT settings. Experimental results demonstrate that ensemble tree-based ML models consistently dominate performance on the engineered tabular feature set as 7 of the top 10 models are ML, with CatBoost and LightGBM achieving near-ceiling accuracy and low false-positive rates. Per-malware analysis further shows that optimal model choice depends on malware behavior. CatBoost is best for Trojan/Spyware, LightGBM for Botnet, XGBoost for Worm, Extra Trees for Rootkit, and Random Forest for Keylogger, while DL models are competitive only for specific categories, with TabNet performing best for Ransomware and FT-Transformer for Virus. In addition, an end-to-end computational time analysis across all 45 models reveals a clear efficiency advantage for boosted tree ensembles relative to most DL architectures, supporting deployment feasibility on commodity CPU hardware. Overall, the study provides actionable guidance for designing adaptive IoT malware detection frameworks, recommending gradient-boosted ensemble ML models as the primary deployment choice, with selective DL models only when category-specific gains justify additional computational cost.