Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selection

TL;DR

This paper introduces ZEUS, a zero-shot method that uses uncertainty estimates to select effective demonstrations for chain-of-thought prompting, improving reasoning performance without requiring model access or extensive human effort.

Contribution

ZEUS is a novel zero-shot demonstration selection strategy based on uncertainty estimates, enhancing chain-of-thought prompting without model access or handcrafted demonstrations.

Findings

ZEUS outperforms existing CoT strategies on four reasoning benchmarks.

ZEUS demonstrates high sensitivity in distinguishing helpful questions.

The method is robust and scalable across different tasks.

Abstract

Chain-of-thought (CoT) prompting has significantly enhanced the capability of large language models (LLMs) by structuring their reasoning processes. However, existing methods face critical limitations: handcrafted demonstrations require extensive human expertise, while trigger phrases are prone to inaccuracies. In this paper, we propose the Zero-shot Uncertainty-based Selection (ZEUS) method, a novel approach that improves CoT prompting by utilizing uncertainty estimates to select effective demonstrations without needing access to model parameters. Unlike traditional methods, ZEUS offers high sensitivity in distinguishing between helpful and ineffective questions, ensuring more precise and reliable selection. Our extensive evaluation shows that ZEUS consistently outperforms existing CoT strategies across four challenging reasoning benchmarks, demonstrating its robustness and scalability.