Hint of Pseudo Code (HoPC): Zero-Shot Step by Step Pseudo Code Reasoning Prompting

TL;DR

This paper introduces HoPC, a novel zero-shot prompting method that enhances multi-step reasoning in large language models by combining problem decomposition, semantic code reasoning, and answer extraction without extra interpreters.

Contribution

HoPC provides a new zero-shot prompting framework that improves reasoning accuracy by integrating pseudo code hints, surpassing existing methods like CoT and PoT in complex tasks.

Findings

HoPC outperforms CoT and PoT on reasoning benchmarks.

HoPC does not require an extra interpreter for code execution.

HoPC demonstrates improved semantic reasoning capabilities.

Abstract

Prompting a language model (LM) is an increasingly important research topic for better utilization of large language models (LLMs). While simple prompting is effective for single-step questions, it fails to activate the correct knowledge path for multi-step reasoning tasks consistently. The few-shot Chain of Thought (CoT), serves as an advanced prompting strategy that explains and demonstrates the reasoning process to the LLM, outperforming simple prompting in challenging reasoning tasks such as arithmetic and common-sense reasoning. The Program of Thought (PoT) aims to generate text and programming language solutions for multi-step reasoning problems. In zero-shot CoT, the prompt is simply ``Let's think step by step'', which is overly simplistic and does not adequately demonstrate a robust reasoning process for complex reasoning challenges. Additionally, PoT requires an extra interpreter to execute the answer and struggles with semantic reasoning problems like StrategyQA. This paper introduces a novel Hint of Pseudo Code (HoPC) prompting technique that does not require extra interpreter as in PoT and incorporates a more powerful zero-shot problem decomposition and semantic code reasoning capabilities than zero-shot CoT. It consists of three components: problem decomposition, semantic code reasoning, and answer extraction. We prompt these components as hints in a sequential, step by step manner, making it easy to tailor and explain for various tasks.