Advancing Software Security and Reliability in Cloud Platforms through AI-based Anomaly Detection

TL;DR

This paper presents an AI-based anomaly detection system using CNN and LSTM to enhance security in CI/CD pipelines and cloud environments, achieving high accuracy in identifying cyber threats.

Contribution

It introduces a novel AI-driven approach combining CNN and LSTM for network traffic anomaly detection within CI/CD workflows, improving security and reliability.

Findings

Achieved over 98% accuracy in detecting network anomalies.

Successfully integrated anomaly detection into CI/CD pipelines.

Demonstrated effectiveness on CSE-CIC-IDS datasets.

Abstract

Continuous Integration/Continuous Deployment (CI/CD) is fundamental for advanced software development, supporting faster and more efficient delivery of code changes into cloud environments. However, security issues in the CI/CD pipeline remain challenging, and incidents (e.g., DDoS, Bot, Log4j, etc.) are happening over the cloud environments. While plenty of literature discusses static security testing and CI/CD practices, only a few deal with network traffic pattern analysis to detect different cyberattacks. This research aims to enhance CI/CD pipeline security by implementing anomaly detection through AI (Artificial Intelligence) support. The goal is to identify unusual behaviour or variations from network traffic patterns in pipeline and cloud platforms. The system shall integrate into the workflow to continuously monitor pipeline activities and cloud infrastructure. Additionally, it aims to explore adaptive response mechanisms to mitigate the detected anomalies or security threats. This research employed two popular network traffic datasets, CSE-CIC-IDS2018 and CSE-CIC-IDS2017. We implemented a combination of Convolution Neural Network(CNN) and Long Short-Term Memory (LSTM) to detect unusual traffic patterns. We achieved an accuracy of 98.69% and 98.30% and generated log files in different CI/CD pipeline stages that resemble the network anomalies affected to address security challenges in modern DevOps practices, contributing to advancing software security and reliability.