You Don't Need Prompt Engineering Anymore: The Prompting Inversion

TL;DR

This paper introduces 'Sculpting', a rule-based prompting method that improves reasoning in mid-tier models but can hinder advanced models, highlighting the need for evolving prompting strategies with model capabilities.

Contribution

The paper presents 'Sculpting', a novel constrained prompting technique that outperforms standard Chain-of-Thought prompting on certain models and analyzes its limitations on more advanced models.

Findings

Sculpting improves performance on gpt-4o (97% vs. 93%)

Sculpting is less effective or detrimental on gpt-5 (94% vs. 96.36%)

Optimal prompting strategies should adapt to model capabilities.

Abstract

Prompt engineering, particularly Chain-of-Thought (CoT) prompting, significantly enhances LLM reasoning capabilities. We introduce "Sculpting," a constrained, rule-based prompting method designed to improve upon standard CoT by reducing errors from semantic ambiguity and flawed common sense. We evaluate three prompting strategies (Zero Shot, standard CoT, and Sculpting) across three OpenAI model generations (gpt-4o-mini, gpt-4o, gpt-5) using the GSM8K mathematical reasoning benchmark (1,317 problems). Our findings reveal a "Prompting Inversion": Sculpting provides advantages on gpt-4o (97% vs. 93% for standard CoT), but becomes detrimental on gpt-5 (94.00% vs. 96.36% for CoT on full benchmark). We trace this to a "Guardrail-to-Handcuff" transition where constraints preventing common-sense errors in mid-tier models induce hyper-literalism in advanced models. Our detailed error analysis demonstrates that optimal prompting strategies must co-evolve with model capabilities, suggesting simpler prompts for more capable models.