CauseJudger: Identifying the Cause with LLMs for Abductive Logical Reasoning

TL;DR

CauseJudger (CJ) leverages large language models to improve abductive logical reasoning by transforming reverse thinking into forward reasoning, effectively removing irrelevant info, and achieving high accuracy with minimal LLM calls.

Contribution

Introduces CauseJudger, a novel framework for abductive reasoning with LLMs, and constructs a large dataset for evaluation, demonstrating significant accuracy improvements.

Findings

CJ achieves up to 41% accuracy improvement with GPT-3.5.

CJ attains over 90% accuracy with GPT-4.

Efficient implementation requiring only two LLM calls.

Abstract

Large language models (LLMs) have been utilized in solving diverse reasoning tasks, encompassing common sense, arithmetic and deduction tasks. However, with difficulties of reversing thinking patterns and irrelevant premises, how to determine the authenticity of the cause in abductive logical reasoning remains underexplored. Inspired by hypothesis and verification method and identification of irrelevant information in human thinking process, we propose a new framework for LLMs abductive logical reasoning called CauseJudger (CJ), which identifies the authenticity of possible cause by transforming thinking from reverse to forward and removing irrelevant information. In addition, we construct an abductive logical reasoning dataset for decision task called CauseLogics, which contains 200,000 tasks of varying reasoning lengths. Our experiments show the efficiency of CJ with overall experiments and ablation experiments as well as case studies on our dataset and reconstructed public dataset. Notably, CJ's implementation is efficient, requiring only two calls to LLM. Its impact is profound: when using gpt-3.5, CJ achieves a maximum correctness improvement of 41% compared to Zero-Shot-CoT. Moreover, with gpt-4, CJ attains an accuracy exceeding 90% across all datasets.