Automatic Chain of Thought Prompting in Large Language Models

TL;DR

Auto-CoT automatically generates diverse reasoning demonstrations for large language models, eliminating manual effort and achieving performance comparable or superior to manually crafted prompts across multiple reasoning benchmarks.

Contribution

The paper introduces Auto-CoT, an automatic method for generating reasoning demonstrations that leverages diversity sampling, reducing manual effort in chain-of-thought prompting.

Findings

Auto-CoT matches or exceeds manual CoT performance on ten benchmarks.

Diversity sampling improves reasoning chain quality.

Auto-CoT reduces manual prompt engineering effort.

Abstract

Large language models (LLMs) can perform complex reasoning by generating intermediate reasoning steps. Providing these steps for prompting demonstrations is called chain-of-thought (CoT) prompting. CoT prompting has two major paradigms. One leverages a simple prompt like "Let's think step by step" to facilitate step-by-step thinking before answering a question. The other uses a few manual demonstrations one by one, each composed of a question and a reasoning chain that leads to an answer. The superior performance of the second paradigm hinges on the hand-crafting of task-specific demonstrations one by one. We show that such manual efforts may be eliminated by leveraging LLMs with the "Let's think step by step" prompt to generate reasoning chains for demonstrations one by one, i.e., let's think not just step by step, but also one by one. However, these generated chains often come with mistakes. To mitigate the effect of such mistakes, we find that diversity matters for automatically constructing demonstrations. We propose an automatic CoT prompting method: Auto-CoT. It samples questions with diversity and generates reasoning chains to construct demonstrations. On ten public benchmark reasoning tasks with GPT-3, Auto-CoT consistently matches or exceeds the performance of the CoT paradigm that requires manual designs of demonstrations. Code is available at https://github.com/amazon-research/auto-cot