Error-Driven Prompt Optimization for Arithmetic Reasoning

TL;DR

This paper presents an error-driven prompt optimization method that significantly improves the arithmetic reasoning accuracy of small language models, making them more reliable and suitable for industrial, privacy-sensitive applications.

Contribution

The paper introduces an iterative, error-driven prompt refinement framework that enhances small language models' arithmetic reasoning without fine-tuning, outperforming larger models in privacy-sensitive settings.

Findings

Model accuracy improved to 70.8% with the proposed method.

Error clustering and prompt refinement significantly boost arithmetic performance.

Small models can surpass larger models like GPT-3.5 Turbo in privacy-preserving scenarios.

Abstract

Recent advancements in artificial intelligence have sparked interest in industrial agents capable of supporting analysts in regulated sectors, such as finance and healthcare, within tabular data workflows. A key capability for such systems is performing accurate arithmetic operations on structured data while ensuring sensitive information never leaves secure, on-premises environments. Here, we introduce an error-driven optimization framework for arithmetic reasoning that enhances a Code Generation Agent (CGA), specifically applied to on-premises small language models (SLMs). Through a systematic evaluation of a leading SLM (Qwen3 4B), we find that while the base model exhibits fundamental limitations in arithmetic tasks, our proposed error-driven method, which clusters erroneous predictions to refine prompt-rules iteratively, dramatically improves performance, elevating the model's accuracy to 70.8\%. Our results suggest that developing reliable, interpretable, and industrially deployable AI assistants can be achieved not only through costly fine-tuning but also via systematic, error-driven prompt optimization, enabling small models to surpass larger language models (GPT-3.5 Turbo) in a privacy-compliant manner.