StackRAG Agent: Improving Developer Answers with Retrieval-Augmented Generation

TL;DR

StackRAG is a novel retrieval-augmented multiagent system that combines Stack Overflow data with large language models to generate more reliable and relevant developer answers, addressing the limitations of search and LLMs.

Contribution

This paper introduces StackRAG, a new multiagent framework that integrates retrieval from Stack Overflow with LLMs to improve answer accuracy and relevance for developers.

Findings

Generated answers are correct and accurate.

Answers are relevant and useful.

StackRAG outperforms standalone LLMs in answer reliability.

Abstract

Developers spend much time finding information that is relevant to their questions. Stack Overflow has been the leading resource, and with the advent of Large Language Models (LLMs), generative models such as ChatGPT are used frequently. However, there is a catch in using each one separately. Searching for answers is time-consuming and tedious, as shown by the many tools developed by researchers to address this issue. On the other, using LLMs is not reliable, as they might produce irrelevant or unreliable answers (i.e., hallucination). In this work, we present StackRAG, a retrieval-augmented Multiagent generation tool based on LLMs that combines the two worlds: aggregating the knowledge from SO to enhance the reliability of the generated answers. Initial evaluations show that the generated answers are correct, accurate, relevant, and useful.