Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation

TL;DR

This paper investigates the robustness of reasoning in large language models during code generation by applying adversarial prompt perturbations, revealing significant fragility and variability in model performance.

Contribution

It introduces Chain-of-Code Collapse, a systematic evaluation framework with adversarial prompt perturbations to test reasoning robustness in LLMs for code tasks.

Findings

Performance drops up to 42.1% with certain perturbations

Model accuracy improves by up to 35.3% with some modifications

Reveals fragility and unpredictability in LLM reasoning systems

Abstract

Large Language Models (LLMs) have achieved remarkable success in tasks requiring complex reasoning, such as code generation, mathematical problem solving, and algorithmic synthesis -- especially when aided by reasoning tokens and Chain-of-Thought prompting. Yet, a core question remains: do these models truly reason, or do they merely exploit shallow statistical patterns? In this paper, we introduce Chain-of-Code Collapse, where we systematically investigate the robustness of reasoning LLMs by introducing a suite of semantically faithful yet adversarially structured prompt perturbations. Our evaluation -- spanning 700 perturbed code generations derived from LeetCode-style problems -- applies transformations such as storytelling reframing, irrelevant constraint injection, example reordering, and numeric perturbation. We observe that while certain modifications severely degrade performance (with accuracy drops up to -42.1%), others surprisingly improve model accuracy by up to 35.3%, suggesting sensitivity not only to semantics but also to surface-level prompt dynamics. These findings expose the fragility and unpredictability of current reasoning systems, underscoring the need for more principles approaches to reasoning alignments and prompting robustness. We release our perturbation datasets and evaluation framework to promote further research in trustworthy and resilient LLM reasoning.