Abductive Reasoning with Syllogistic Forms in Large Language Models

TL;DR

This paper investigates the ability of large language models to perform abductive reasoning using syllogistic forms, highlighting their potential and biases in contextualized inference beyond formal deduction.

Contribution

It introduces a syllogistic dataset adapted for abductive reasoning and evaluates LLMs' performance, revealing biases and areas for improvement in complex reasoning tasks.

Findings

LLMs show some capability in abductive reasoning with syllogistic forms

Biases similar to human reasoning patterns are observed in LLMs

Potential for enhancing LLMs' reasoning through contextual understanding

Abstract

Research in AI using Large-Language Models (LLMs) is rapidly evolving, and the comparison of their performance with human reasoning has become a key concern. Prior studies have indicated that LLMs and humans share similar biases, such as dismissing logically valid inferences that contradict common beliefs. However, criticizing LLMs for these biases might be unfair, considering our reasoning not only involves formal deduction but also abduction, which draws tentative conclusions from limited information. Abduction can be regarded as the inverse form of syllogism in its basic structure, that is, a process of drawing a minor premise from a major premise and conclusion. This paper explores the accuracy of LLMs in abductive reasoning by converting a syllogistic dataset into one suitable for abduction. It aims to investigate whether the state-of-the-art LLMs exhibit biases in abduction and to identify potential areas for improvement, emphasizing the importance of contextualized reasoning beyond formal deduction. This investigation is vital for advancing the understanding and application of LLMs in complex reasoning tasks, offering insights into bridging the gap between machine and human cognition.