Recursive Neural Networks Can Learn Logical Semantics

TL;DR

This paper investigates whether recursive neural networks can learn logical semantics and perform tasks like entailment and contradiction detection, demonstrating their potential for logical inference in natural language.

Contribution

It evaluates the ability of TreeRNNs and TreeRNTNs to learn logical relationships, showing they can handle logical inference tasks in artificial and natural language data.

Findings

Both models perform well on SICK data.

Models generalize effectively on simulated data.

They can learn representations suitable for logical inference.

Abstract

Tree-structured recursive neural networks (TreeRNNs) for sentence meaning have been successful for many applications, but it remains an open question whether the fixed-length representations that they learn can support tasks as demanding as logical deduction. We pursue this question by evaluating whether two such models---plain TreeRNNs and tree-structured neural tensor networks (TreeRNTNs)---can correctly learn to identify logical relationships such as entailment and contradiction using these representations. In our first set of experiments, we generate artificial data from a logical grammar and use it to evaluate the models' ability to learn to handle basic relational reasoning, recursive structures, and quantification. We then evaluate the models on the more natural SICK challenge data. Both models perform competitively on the SICK data and generalize well in all three experiments on simulated data, suggesting that they can learn suitable representations for logical inference in natural language.