SCREWS: A Modular Framework for Reasoning with Revisions

TL;DR

SCREWS is a modular framework that enhances reasoning with LLMs by enabling diverse revision strategies and effective selection, improving performance across various reasoning tasks.

Contribution

It introduces a flexible, modular framework for reasoning with revisions, unifying previous methods and enabling novel strategies for error correction.

Findings

Heterogeneous revision strategies improve reasoning accuracy.

Selection between original and revised outputs is crucial.

Framework achieves state-of-the-art results on multiple tasks.

Abstract

Large language models (LLMs) can improve their accuracy on various tasks through iteratively refining and revising their output based on feedback. We observe that these revisions can introduce errors, in which case it is better to roll back to a previous result. Further, revisions are typically homogeneous: they use the same reasoning method that produced the initial answer, which may not correct errors. To enable exploration in this space, we present SCREWS, a modular framework for reasoning with revisions. It is comprised of three main modules: Sampling, Conditional Resampling, and Selection, each consisting of sub-modules that can be hand-selected per task. We show that SCREWS not only unifies several previous approaches under a common framework, but also reveals several novel strategies for identifying improved reasoning chains. We evaluate our framework with state-of-the-art LLMs (ChatGPT and GPT-4) on a diverse set of reasoning tasks and uncover useful new reasoning strategies for each: arithmetic word problems, multi-hop question answering, and code debugging. Heterogeneous revision strategies prove to be important, as does selection between original and revised candidates.