On Iterative Evaluation and Enhancement of Code Quality Using GPT-4o

TL;DR

CodeQUEST is a framework that uses GPT-4o to iteratively evaluate and improve code quality across multiple dimensions, demonstrating significant enhancements and close alignment with established metrics.

Contribution

Introduces CodeQUEST, a novel LLM-based framework for iterative code evaluation and enhancement across multiple quality dimensions, validated with experiments on Python and JavaScript.

Findings

Achieved a 52.6% mean relative improvement in code quality.

Evaluations closely align with established metrics like Pylint and Radon.

Demonstrated robustness and effectiveness in automated code quality enhancement.

Abstract

This paper introduces CodeQUEST, a novel framework leveraging Large Language Models (LLMs) to iteratively evaluate and enhance code quality across multiple dimensions, including readability, maintainability, efficiency, and security. The framework is divided into two main components: an Evaluator that assesses code quality across ten dimensions, providing both quantitative scores and qualitative summaries, and an Optimizer that iteratively improves the code based on the Evaluator's feedback. Our study demonstrates that CodeQUEST can effectively and robustly evaluate code quality, with its assessments aligning closely with established code quality metrics. Through a series of experiments using a curated dataset of Python and JavaScript examples, CodeQUEST demonstrated significant improvements in code quality, achieving a mean relative percentage improvement of 52.6%. The framework's evaluations were validated against a set of proxy metrics comprising of Pylint Score, Radon Maintainability Index, and Bandit output logs, showing a meaningful correlation. This highlights the potential of LLMs in automating code quality evaluation and improvement processes, presenting a significant advancement toward enhancing software development practices. The code implementation of the framework is available at: https://github.com/jpmorganchase/CodeQuest.