Effects of structure on reasoning in instance-level Self-Discover

TL;DR

This paper compares structured and unstructured reasoning in large language models, finding unstructured approaches generally outperform structured ones, especially on complex benchmarks, and highlights the importance of reasoning granularity.

Contribution

It introduces iSelf-Discover, an instance-level framework for dynamic reasoning plan generation, and empirically demonstrates the advantages of unstructured reasoning over structured formats.

Findings

Unstructured reasoning outperforms structured reasoning on diverse benchmarks.

Unstructured plans achieve up to 18.90% better performance on the MATH benchmark.

Zero-shot unstructured variants outperform five-shot structured counterparts.

Abstract

The drive for predictable LLM reasoning in their integration with compound systems has popularized structured outputs, yet concerns remain about performance trade-offs compared to unconstrained natural language. At the same time, training on unconstrained Chain of Thought (CoT) traces has brought about a new class of strong reasoning models that nevertheless present novel compute budget and faithfulness challenges. This paper introduces iSelf-Discover, an instance-level adaptation of the Self-Discover framework, and using it compares dynamically generated structured JSON reasoning with its unstructured counterpart. Our empirical evaluation across diverse benchmarks using state-of-the-art open-source models supports a consistent advantage for unstructured reasoning. Notably, on the complex MATH benchmark, unstructured plans achieved relative performance improvements of up to 18.90\% over structured approaches. Zero-shot unstructured iSelf-Discover variants are also shown to outperform their five-shot structured counterparts, underscoring the significance of this gap, even when structured plans are dynamically generated to ensure reasoning precedes the final answer. We further demonstrate that the optimal granularity of plan generation (instance-level vs. task-level) is context-dependent. These findings invite re-evaluation of the reliance on structured formats for complex problem-solving and how compound systems should be organized.