Are LLMs Correctly Integrated into Software Systems?

TL;DR

This paper investigates the challenges of integrating large language models with retrieval-augmented generation into software systems, identifying common defect patterns and proposing guidelines to improve integration quality.

Contribution

It provides a comprehensive analysis of 100 open-source applications, identifies 18 defect patterns, and offers systematic guidelines and an open-source defect library for better LLM integration.

Findings

77% of applications have more than three types of integration defects

Identified 18 common defect patterns affecting functionality, efficiency, and security

Proposed systematic guidelines and developed the Hydrangea defect library

Abstract

Large language models (LLMs) provide effective solutions in various application scenarios, with the support of retrieval-augmented generation (RAG). However, developers face challenges in integrating LLM and RAG into software systems, due to lacking interface specifications, various requirements from software context, and complicated system management. In this paper, we have conducted a comprehensive study of 100 open-source applications that incorporate LLMs with RAG support, and identified 18 defect patterns. Our study reveals that 77% of these applications contain more than three types of integration defects that degrade software functionality, efficiency, and security. Guided by our study, we propose systematic guidelines for resolving these defects in software life cycle. We also construct an open-source defect library Hydrangea.