Improving Predictability of User-Affecting Metrics to Support Anomaly Detection in Cloud Services

TL;DR

This paper explores how to improve anomaly detection in cloud services by balancing system performance and security, using analytical modeling and load testing to optimize detection configurations.

Contribution

It introduces a combined analytical and experimental approach with a heavy-tail bi-modal model to optimize IDS configurations for better security and performance trade-offs.

Findings

Optimal number of servers minimizes response time variance.

Trade-off between response time and classification accuracy.

Security benefits may justify slight performance sacrifices.

Abstract

Anomaly detection systems aim to detect and report attacks or unexpected behavior in networked systems. Previous work has shown that anomalies have an impact on system performance, and that performance signatures can be effectively used for implementing an IDS. In this paper, we present an analytical and an experimental study on the trade-off between anomaly detection based on performance signatures and system scalability. The proposed approach combines analytical modeling and load testing to find optimal configurations for the signature-based IDS. We apply a heavy-tail bi-modal modeling approach, where "long" jobs represent large resource consuming transactions, e.g., generated by DDoS attacks; the model was parametrized using results obtained from controlled experiments. For performance purposes, mean response time is the key metric to be minimized, whereas for security purposes, response time variance and classification accuracy must be taken into account. The key insights from our analysis are: (i) there is an optimal number of servers which minimizes the response time variance, (ii) the sweet-spot number of servers that minimizes response time variance and maximizes classification accuracy is typically smaller than or equal to the one that minimizes mean response time. Therefore, for security purposes, it may be worth slightly sacrificing performance to increase classification accuracy.