The Last Dependency Crusade: Solving Python Dependency Conflicts with LLMs

TL;DR

This paper introduces PLLM, a retrieval-augmented generation approach using Large Language Models to automatically repair Python dependency issues, significantly outperforming existing methods on real-world datasets.

Contribution

The paper presents PLLM, a novel LLM-based method with RAG for fixing Python dependency conflicts, demonstrating superior performance over state-of-the-art baselines.

Findings

RAG improves fix rates across multiple LLMs.

PLLM outperforms PyEGo and ReadPyE in fix success.

Effective for projects with many dependencies and specialized libraries.

Abstract

Resolving Python dependency issues remains a tedious and error-prone process, forcing developers to manually trial compatible module versions and interpreter configurations. Existing automated solutions, such as knowledge-graph-based and database-driven methods, face limitations due to the variety of dependency error types, large sets of possible module versions, and conflicts among transitive dependencies. This paper investigates the use of Large Language Models (LLMs) to automatically repair dependency issues in Python programs. We propose PLLM (pronounced "plum"), a novel retrieval-augmented generation (RAG) approach that iteratively infers missing or incorrect dependencies. PLLM builds a test environment where the LLM proposes module combinations, observes execution feedback, and refines its predictions using natural language processing (NLP) to parse error messages. We evaluate PLLM on the Gistable HG2.9K dataset, a curated collection of real-world Python programs. Using this benchmark, we explore multiple PLLM configurations, including six open-source LLMs evaluated both with and without RAG. Our findings show that RAG consistently improves fix rates, with the best performance achieved by Gemma-2 9B when combined with RAG. Compared to two state-of-the-art baselines, PyEGo and ReadPyE, PLLM achieves significantly higher fix rates; +15.97\% more than ReadPyE and +21.58\% more than PyEGo. Further analysis shows that PLLM is especially effective for projects with numerous dependencies and those using specialized numerical or machine-learning libraries.