ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting

TL;DR

ChainLM introduces a novel framework for automatically generating high-quality chain-of-thought prompts, improving reasoning in large language models through evolutionary strategies and a step-level debating method, leading to better performance on complex tasks.

Contribution

The paper presents CoTGenius, an innovative framework for automatic CoT prompt generation, and introduces ChainLM, a fine-tuned Llama 2 model with enhanced reasoning capabilities.

Findings

ChainLM outperforms existing models on complex reasoning tasks.

The step-level debating method reduces cumulative reasoning errors.

Data category analysis improves understanding of prompt effectiveness.

Abstract

Chain-of-Thought (CoT) prompting can enhance the reasoning capabilities of large language models (LLMs), establishing itself as a primary approach to solving complex reasoning tasks. Existing CoT synthesis approaches usually focus on simpler reasoning tasks and thus result in low-quality and inconsistent CoT prompts. In response to this challenge, we present an empirical investigation of CoT prompting and introduce CoTGenius, a novel framework designed for the automatic generation of superior CoT prompts. CoTGenius is developed based on three major evolution strategies, i.e., complicate, diversify, and specify-alongside two filtering mechanisms: evolutionary success judgement and correctness verification. We further employ CoTGenius to create an extensive CoT dataset, and subsequently fine-tune the Llama 2-Chat 7B and 13B models on this dataset. We call the resulting model ChainLM. To deal with the cumulative error issue in reasoning steps, we propose a step-level debating method, wherein multiple debaters discuss each reasoning step to arrive at the correct answer. Extensive experiments demonstrate that our ChainLM models exhibit enhanced proficiency in addressing a spectrum of complex reasoning problems compared to existing models. In addition, we conduct an in-depth analysis of the impact of data categories within CoTGenius on the model performance. We release our dataset and code at https://github.com/RUCAIBox/ChainLM.