Can Transformers Reason in Fragments of Natural Language?

TL;DR

This paper empirically investigates whether transformer models genuinely reason in natural language fragments, finding they overfit superficial patterns despite high performance, thus questioning their true reasoning capabilities.

Contribution

It provides a large-scale empirical analysis of transformer models' reasoning abilities in complex natural language fragments, highlighting their tendency to overfit superficial cues.

Findings

Transformers perform well on reasoning tasks in natural language fragments.

They tend to overfit superficial patterns rather than learn logical principles.

This raises questions about the true reasoning capabilities of current NLP models.

Abstract

State-of-the-art deep-learning-based approaches to Natural Language Processing (NLP) are credited with various capabilities that involve reasoning with natural language texts. In this paper we carry out a large-scale empirical study investigating the detection of formally valid inferences in controlled fragments of natural language for which the satisfiability problem becomes increasingly complex. We find that, while transformer-based language models perform surprisingly well in these scenarios, a deeper analysis re-veals that they appear to overfit to superficial patterns in the data rather than acquiring the logical principles governing the reasoning in these fragments.