AbductionRules: Training Transformers to Explain Unexpected Inputs

TL;DR

This paper introduces AbductionRules, a set of natural language datasets for training and testing Transformers on abductive reasoning, demonstrating their ability to learn generalizable abductive techniques from natural language knowledge bases.

Contribution

The paper presents AbductionRules datasets and shows how pretrained Transformers can be finetuned to perform abductive reasoning over natural language knowledge bases.

Findings

Transformers learned generalizable abductive reasoning techniques.

Models exploited data structure to improve performance.

Discussion on future improvements for abductive reasoning models.

Abstract

Transformers have recently been shown to be capable of reliably performing logical reasoning over facts and rules expressed in natural language, but abductive reasoning - inference to the best explanation of an unexpected observation - has been underexplored despite significant applications to scientific discovery, common-sense reasoning, and model interpretability. We present AbductionRules, a group of natural language datasets designed to train and test generalisable abduction over natural-language knowledge bases. We use these datasets to finetune pretrained Transformers and discuss their performance, finding that our models learned generalisable abductive techniques but also learned to exploit the structure of our data. Finally, we discuss the viability of this approach to abductive reasoning and ways in which it may be improved in future work.