A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?

TL;DR

This paper investigates non-syntactic mistakes made by Large Language Models in code generation, analyzing their causes and exploring GPT-4's effectiveness in mistake detection to improve code quality.

Contribution

It identifies seven categories of non-syntactic mistakes in LLM-generated code, including four previously overlooked, and proposes reasons and detection methods for these errors.

Findings

Seven categories of non-syntactic mistakes identified

GPT-4 with ReAct achieves up to 0.65 F1 in mistake detection

Four new mistake types discovered beyond previous research

Abstract

Recent advancements in Large Language Models (LLMs) have led to their widespread application in automated code generation. However, these models can still generate defective code that deviates from the specification. Previous research has mainly focused on the mistakes in LLM-generated standalone functions, overlooking real-world software development situations where the successful generation of the code requires software contexts such as external dependencies. In this paper, we considered both of these code generation situations and identified a range of \textit{non-syntactic mistakes} arising from LLMs' misunderstandings of coding question specifications. Seven categories of non-syntactic mistakes were identified through extensive manual analyses, four of which were missed by previous works. To better understand these mistakes, we proposed six reasons behind these mistakes from various perspectives. Moreover, we explored the effectiveness of LLMs in detecting mistakes and their reasons. Our evaluation demonstrated that GPT-4 with the ReAct prompting technique can achieve an F1 score of up to 0.65 when identifying reasons for LLM's mistakes, such as misleading function signatures. We believe that these findings offer valuable insights into enhancing the quality of LLM-generated code.