Logical forms complement probability in understanding language model (and human) performance

TL;DR

This paper investigates how large language models perform logical reasoning in natural language, emphasizing the importance of logical forms alongside probability, and compares their reasoning abilities with humans using a new dataset.

Contribution

It introduces a controlled dataset for logical reasoning in LLMs and highlights the significance of logical forms in predicting model behavior, advancing understanding of LLM reasoning capabilities.

Findings

Logical forms significantly influence LLM reasoning performance.

LLMs show both similarities and differences with humans in logical reasoning.

Logical reasoning in LLMs is affected by input logical structure, not just probability.

Abstract

With the increasing interest in using large language models (LLMs) for planning in natural language, understanding their behaviors becomes an important research question. This work conducts a systematic investigation of LLMs' ability to perform logical reasoning in natural language. We introduce a controlled dataset of hypothetical and disjunctive syllogisms in propositional and modal logic and use it as the testbed for understanding LLM performance. Our results lead to novel insights in predicting LLM behaviors: in addition to the probability of input (Gonen et al., 2023; McCoy et al., 2024), logical forms should be considered as important factors. In addition, we show similarities and discrepancies between the logical reasoning performances of humans and LLMs by collecting and comparing behavioral data from both.