A Multi-AI Agent System for Autonomous Optimization of Agentic AI Solutions via Iterative Refinement and LLM-Driven Feedback Loops

TL;DR

This paper presents an autonomous framework for optimizing Agentic AI systems using iterative feedback loops and LLMs, reducing manual tuning and improving performance across various industries.

Contribution

It introduces a novel multi-agent system that autonomously refines AI solutions through LLM-driven feedback loops, enhancing scalability and adaptability without human intervention.

Findings

Significant improvements in output quality and relevance.

Effective autonomous optimization across multiple domains.

Demonstrated scalability and robustness in real-world scenarios.

Abstract

Agentic AI systems use specialized agents to handle tasks within complex workflows, enabling automation and efficiency. However, optimizing these systems often requires labor-intensive, manual adjustments to refine roles, tasks, and interactions. This paper introduces a framework for autonomously optimizing Agentic AI solutions across industries, such as NLP-driven enterprise applications. The system employs agents for Refinement, Execution, Evaluation, Modification, and Documentation, leveraging iterative feedback loops powered by an LLM (Llama 3.2-3B). The framework achieves optimal performance without human input by autonomously generating and testing hypotheses to improve system configurations. This approach enhances scalability and adaptability, offering a robust solution for real-world applications in dynamic environments. Case studies across diverse domains illustrate the transformative impact of this framework, showcasing significant improvements in output quality, relevance, and actionability. All data for these case studies, including original and evolved agent codes, along with their outputs, are here: https://anonymous.4open.science/r/evolver-1D11/