MoA is All You Need: Building LLM Research Team using Mixture of Agents

TL;DR

This paper introduces the Mixture of Agents (MoA) framework, a layered network of small language models designed for retrieval-augmented generation in finance, demonstrating improved response quality and cost efficiency.

Contribution

The paper presents the MoA framework as a practical, customizable approach for scaling RAG applications, with real-world evaluation in financial domains.

Findings

MoA produces higher quality, more grounded responses.

MoA maintains low operational costs.

Effective across various financial tasks.

Abstract

Large Language Models (LLMs) research in the financial domain is particularly complex due to the sheer number of approaches proposed in literature. Retrieval-Augmented Generation (RAG) has emerged as one of the leading methods in the sector due to its inherent groundedness and data source variability. In this work, we introduce a RAG framework called Mixture of Agents (MoA) and demonstrate its viability as a practical, customizable, and highly effective approach for scaling RAG applications. MoA is essentially a layered network of individually customized small language models (Hoffmann et al., 2022) collaborating to answer questions and extract information. While there are many theoretical propositions for such an architecture and even a few libraries for generally applying the structure in practice, there are limited documented studies evaluating the potential of this framework considering real business constraints such as cost and speed. We find that the MoA framework, consisting of small language models (Hoffmann et al., 2022), produces higher quality and more grounded responses across various financial domains that are core to Vanguard's business while simultaneously maintaining low costs.