Augmenting Large Language Models with Static Code Analysis for Automated Code Quality Improvements

TL;DR

This paper presents a novel approach combining static code analysis, large language models, and retrieval-augmented generation to automate code issue detection and correction, significantly improving code quality and development efficiency.

Contribution

It introduces an integrated framework that leverages static analysis, LLMs, and RAG to automate code revisions and address hallucinations, advancing automated software maintenance.

Findings

Significant reduction in code issues after applying the framework

Enhanced accuracy of code revisions through retrieval-augmented generation

Effective mitigation of LLM hallucinations with a custom comparison tool

Abstract

This study examined code issue detection and revision automation by integrating Large Language Models (LLMs) such as OpenAI's GPT-3.5 Turbo and GPT-4o into software development workflows. A static code analysis framework detects issues such as bugs, vulnerabilities, and code smells within a large-scale software project. Detailed information on each issue was extracted and organized to facilitate automated code revision using LLMs. An iterative prompt engineering process is applied to ensure that prompts are structured to produce accurate and organized outputs aligned with the project requirements. Retrieval-augmented generation (RAG) is implemented to enhance the relevance and precision of the revisions, enabling LLM to access and integrate real-time external knowledge. The issue of LLM hallucinations - where the model generates plausible but incorrect outputs - is addressed by a custom-built "Code Comparison App," which identifies and corrects erroneous changes before applying them to the codebase. Subsequent scans using the static code analysis framework revealed a significant reduction in code issues, demonstrating the effectiveness of combining LLMs, static analysis, and RAG to improve code quality, streamline the software development process, and reduce time and resource expenditure.