Non-Interactive Symbolic-Aided Chain-of-Thought for Logical Reasoning

TL;DR

This paper proposes Symbolic-Aided Chain-of-Thought, integrating symbolic representations into LLM reasoning prompts to improve transparency, interpretability, and performance on logical reasoning benchmarks.

Contribution

It introduces a novel non-interactive method that combines symbolic structures with chain-of-thought prompting to enhance logical reasoning in large language models.

Findings

Significantly improves reasoning accuracy on multiple benchmarks.

Enhances interpretability and transparency of LLM reasoning processes.

Outperforms standard CoT on three of four datasets.

Abstract

This work introduces Symbolic-Aided Chain-of-Thought (CoT), an improved approach to standard CoT, for logical reasoning in large language models (LLMs). The key idea is to integrate lightweight symbolic representations into few-shot prompts, structuring the inference steps with a consistent strategy to make reasoning patterns more explicit within a non-interactive reasoning process. By incorporating these symbolic structures, Symbolic-Aided CoT preserves the generalizability of standard prompting techniques while enhancing the transparency, interpretability, and analyzability of LLM logical reasoning. Extensive experiments on four well-known logical reasoning benchmarks -- ProofWriter, FOLIO, ProntoQA, and LogicalDeduction, which cover diverse reasoning tasks and scenarios -- demonstrate the effectiveness of the proposed approach, particularly in complex reasoning tasks that require navigating multiple constraints or rules. Notably, Symbolic-Aided CoT consistently improves LLMs' reasoning capabilities across various model sizes and significantly outperforms conventional CoT on three out of four datasets, ProofWriter, ProntoQA, and LogicalDeduction.