Learning to Reason with Third-Order Tensor Products

TL;DR

This paper introduces a neural architecture combining RNNs with tensor product representations to enhance symbolic reasoning and systematic generalization in natural language tasks, outperforming state-of-the-art models.

Contribution

It presents a novel end-to-end trainable model that integrates tensor product representations with RNNs for improved reasoning capabilities.

Findings

Outperforms state-of-the-art models on reasoning tasks

Shows better generalization with systematic data differences

Achieves significant improvements in both single-task and multi-task settings

Abstract

We combine Recurrent Neural Networks with Tensor Product Representations to learn combinatorial representations of sequential data. This improves symbolic interpretation and systematic generalisation. Our architecture is trained end-to-end through gradient descent on a variety of simple natural language reasoning tasks, significantly outperforming the latest state-of-the-art models in single-task and all-tasks settings. We also augment a subset of the data such that training and test data exhibit large systematic differences and show that our approach generalises better than the previous state-of-the-art.