Watch Your Steps: Observable and Modular Chains of Thought

TL;DR

This paper introduces Program Trace Prompting, a method that formalizes chain of thought explanations using Python syntax to improve observability and enable analysis, achieving strong results across diverse tasks.

Contribution

It presents a novel formalization of chain of thought prompting that enhances observability and analysis, addressing non-local errors and modularity verification.

Findings

Strong performance on 23 BIG-Bench Hard tasks

Identification of non-local errors in reasoning

Methods for verifying step modularity

Abstract

We propose a variant of chain of thought (CoT) prompting called Program Trace Prompting that makes explanations more observable while preserving the power, generality and flexibility of CoT. In our approach, few-shot CoT demonstrations are wrapped in a formal syntax based on Python, and each prompt: identifies and names steps; defines the input/output behavior of steps; and replaces CoT explanations of in-context examples with chains of these formalized steps on the same examples. Program Trace Prompting is applicable to many tasks, achieving strong results on the 23 diverse tasks in the BIG-Bench Hard benchmark. More importantly, by instrumenting explanations in this way, we enable new types of analysis. In particular, we identify "non-local errors" (which correspond to incorrectly learning the reasoning method illustrated in the demonstrations) as an unaddressed issue in CoT learning, and we present methods for verifying the modularity of steps in a CoT explanation.