DepsRAG: Towards Agentic Reasoning and Planning for Software Dependency Management

TL;DR

DepsRAG is a multi-agent framework leveraging LLMs, knowledge graphs, and retrieval-augmented generation to assist developers in reasoning about software dependencies, improving accuracy through a Critic-Agent feedback loop.

Contribution

This paper introduces DepsRAG, a novel multi-agent system that combines knowledge graphs, retrieval-augmented generation, and feedback mechanisms to enhance software dependency reasoning.

Findings

Threefold increase in reasoning accuracy with Critic-Agent

Effective integration of KG and external sources for dependency analysis

Demonstrated adaptability to complex dependency scenarios

Abstract

In the era of Large Language Models (LLMs) with their advanced capabilities, a unique opportunity arises to develop LLM-based digital assistant tools that can support software developers by facilitating comprehensive reasoning about software dependencies and open-source libraries before importing them. This reasoning process is daunting, mandating multiple specialized tools and dedicated expertise, each focusing on distinct aspects (e.g., security analysis tools may overlook design flaws such as circular dependencies, which hinder software maintainability). Creating a significant bottleneck in the software development lifecycle. In this paper, we introduce DepsRAG, a multi-agent framework designed to assist developers in reasoning about software dependencies. DepsRAG first constructs a comprehensive Knowledge Graph (KG) that includes both direct and transitive dependencies. Developers can interact with DepsRAG through a conversational interface, posing queries about the dependencies. DepsRAG employs Retrieval-Augmented Generation (RAG) to enhance these queries by retrieving relevant information from the KG as well as external sources, such as the Web and vulnerability databases, thus demonstrating its adaptability to novel scenarios. DepsRAG incorporates a Critic-Agent feedback loop to ensure the accuracy and clarity of LLM-generated responses. We evaluated DepsRAG using GPT-4-Turbo and Llama-3 on three multi-step reasoning tasks, observing a threefold increase in accuracy with the integration of the Critic-Agent mechanism. DepsRAG demo and implementation are available: https://github.com/Mohannadcse/DepsRAG.