Engineering LLM Powered Multi-agent Framework for Autonomous CloudOps

TL;DR

This paper presents MOYA, a multi-agent framework leveraging GenAI and RAG to automate and optimize CloudOps tasks, improving accuracy and responsiveness in complex cloud management workflows.

Contribution

Introduction of MOYA, a novel multi-agent framework that integrates GenAI and RAG for autonomous CloudOps with balanced human oversight.

Findings

Enhanced accuracy and responsiveness over non-agentic approaches

Effective handling of complex workflows and diverse data sources

Improved automation and reliability in cloud management tasks

Abstract

Cloud Operations (CloudOps) is a rapidly growing field focused on the automated management and optimization of cloud infrastructure which is essential for organizations navigating increasingly complex cloud environments. MontyCloud Inc. is one of the major companies in the CloudOps domain that leverages autonomous bots to manage cloud compliance, security, and continuous operations. To make the platform more accessible and effective to the customers, we leveraged the use of GenAI. Developing a GenAI-based solution for autonomous CloudOps for the existing MontyCloud system presented us with various challenges such as i) diverse data sources; ii) orchestration of multiple processes; and iii) handling complex workflows to automate routine tasks. To this end, we developed MOYA, a multi-agent framework that leverages GenAI and balances autonomy with the necessary human control. This framework integrates various internal and external systems and is optimized for factors like task orchestration, security, and error mitigation while producing accurate, reliable, and relevant insights by utilizing Retrieval Augmented Generation (RAG). Evaluations of our multi-agent system with the help of practitioners as well as using automated checks demonstrate enhanced accuracy, responsiveness, and effectiveness over non-agentic approaches across complex workflows.