A Quantized VAE-MLP Botnet Detection Model: A Systematic Evaluation of Quantization-Aware Training and Post-Training Quantization Strategies

TL;DR

This paper evaluates quantization strategies for a VAE-MLP botnet detection model, demonstrating that post-training quantization offers significant size and speed benefits with minimal accuracy loss, enhancing deployment on resource-limited IoT devices.

Contribution

It systematically compares quantization-aware training and post-training quantization for a VAE-MLP botnet detection model, highlighting PTQ's efficiency and minimal impact on detection accuracy.

Findings

PTQ achieves 6x speedup and 21x size reduction with minimal accuracy loss.

QAT results in greater accuracy decline but still offers 3x speedup and 24x compression.

Both strategies enable lightweight, efficient IoT botnet detection models.

Abstract

In an effort to counter the increasing IoT botnet-based attacks, state-of-the-art deep learning methods have been proposed and have achieved impressive detection accuracy. However, their computational intensity restricts deployment on resource-constrained IoT devices, creating a critical need for lightweight detection models. A common solution to this challenge is model compression via quantization. This study proposes a VAE-MLP model framework where an MLP-based classifier is trained on 8-dimensional latent vectors derived from the high-dimensional train data using the encoder component of a pretrained variational autoencoder (VAE). Two widely used quantization strategies--Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ)--are then systematically evaluated in terms of their impact on detection performance, storage efficiency, and inference latency using two benchmark IoT botnet datasets--N-BaIoT and CICIoT2022. The results revealed that, with respect to detection accuracy, the QAT strategy experienced a more noticeable decline,whereas PTQ incurred only a marginal reduction compared to the original unquantized model. Furthermore, PTQ yielded a 6x speedup and 21x reduction in size, while QAT achieved a 3x speedup and 24x compression, demonstrating the practicality of quantization for device-level IoT botnet detection.