Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework

TL;DR

This paper introduces FPBench, a novel evaluation framework for assessing how large language models handle faulty premises in code generation, revealing their limited self-scrutiny and reasoning abilities under such conditions.

Contribution

It presents the first systematic framework for evaluating LLMs with faulty premises, including constructing premise categories and multi-dimensional metrics, to analyze model deficiencies.

Findings

Most models perform poorly with faulty premises.

Increasing prompt length does not improve code quality under faults.

Distinct faulty premise types activate different model defect patterns.

Abstract

With the advancement of code generation capabilities in large language models (LLMs), their reliance on input premises has intensified. When users provide inputs containing faulty premises, the probability of code generation hallucinations rises significantly, exposing deficiencies in their self-scrutiny capabilities. This paper proposes Faulty Premises Bench (FPBench), the first code generation evaluation framework targeting faulty premises. By systematically constructing three categories of faulty premises and integrating multi-dimensional evaluation metrics, it conducts in-depth assessments of 15 representative LLMs. The key findings are as follows: (1) Most models exhibit poor reasoning abilities and suboptimal code generation performance under faulty premises, heavily relying on explicit prompts for error detection, with limited self-scrutiny capabilities; (2) Faulty premises trigger a point of diminishing returns in resource investment, leading to blindly increasing length fails to enhance quality; (3) The three types of faulty premises respectively activate distinct defect patterns in models, revealing a triple dissociation in the cognitive mechanisms of code generation models. This study not only highlights the urgent need for LLMs to proactively verify premises in code generation but also, through the proposed FPBench framework and multi-dimensional evaluation system, provides a theoretical foundation and practical pathway for developing reliable, human-centric code generation models.