SQuARE: Sequential Question Answering Reasoning Engine for Enhanced Chain-of-Thought in Large Language Models

TL;DR

SQuARE is a novel prompting technique for large language models that enhances reasoning by systematically decomposing questions into auxiliary queries, leading to improved performance over traditional chain-of-thought methods.

Contribution

Introduces SQuARE, a self-interrogation based prompting method that systematically decomposes complex questions to improve reasoning in large language models.

Findings

SQuARE outperforms traditional CoT prompts on multiple datasets.

SQuARE significantly improves reasoning capabilities in Llama 3 and GPT-4o.

Systematic question decomposition enhances model understanding and accuracy.

Abstract

In the rapidly evolving field of Natural Language Processing, Large Language Models (LLMs) are tasked with increasingly complex reasoning challenges. Traditional methods like chain-of-thought prompting have shown promise but often fall short in fully leveraging a model's reasoning capabilities. This paper introduces SQuARE (Sequential Question Answering Reasoning Engine), a novel prompting technique designed to improve reasoning through a self-interrogation paradigm. Building upon CoT frameworks, SQuARE prompts models to generate and resolve multiple auxiliary questions before tackling the main query, promoting a more thorough exploration of various aspects of a topic. Our expansive evaluations, conducted with Llama 3 and GPT-4o models across multiple question-answering datasets, demonstrate that SQuARE significantly surpasses traditional CoT prompts and existing rephrase-and-respond methods. By systematically decomposing queries, SQuARE advances LLM capabilities in reasoning tasks. The code is publicly available at https://github.com/IntelLabs/RAG-FiT/tree/square.