Are GNNs Worth the Effort for IoT Botnet Detection? A Comparative Study of VAE-GNN vs. ViT-MLP and VAE-MLP Approaches

TL;DR

This study compares the effectiveness of VAE-GNN, ViT-MLP, and VAE-MLP architectures for IoT botnet detection, finding high accuracy in binary tasks but lower performance of GNN models in multiclass classification.

Contribution

It provides a comprehensive evaluation of deep learning architectures, highlighting the comparative performance of GNNs, ViT, and MLP in IoT botnet detection tasks.

Findings

All models achieved over 99.93% accuracy in binary classification.

GNN-based models performed significantly worse in multiclass classification.

VAE-MLP and ViT-MLP achieved the highest accuracy in multiclass tasks.

Abstract

Due to the exponential rise in IoT-based botnet attacks, researchers have explored various advanced techniques for both dimensionality reduction and attack detection to enhance IoT security. Among these, Variational Autoencoders (VAE), Vision Transformers (ViT), and Graph Neural Networks (GNN), including Graph Convolutional Networks (GCN) and Graph Attention Networks (GAT), have garnered significant research attention in the domain of attack detection. This study evaluates the effectiveness of four state-of-the-art deep learning architectures for IoT botnet detection: a VAE encoder with a Multi-Layer Perceptron (MLP), a VAE encoder with a GCN, a VAE encoder with a GAT, and a ViT encoder with an MLP. The evaluation is conducted on a widely studied IoT benchmark dataset--the N-BaIoT dataset for both binary and multiclass tasks. For the binary classification task, all models achieved over 99.93% in accuracy, recall, precision, and F1-score, with no notable differences in performance. In contrast, for the multiclass classification task, GNN-based models showed significantly lower performance compared to VAE-MLP and ViT-MLP, with accuracies of 86.42%, 89.46%, 99.72%, and 98.38% for VAE-GCN, VAE-GAT, VAE-MLP, and ViT-MLP, respectively.