Tensor Product Generation Networks for Deep NLP Modeling

TL;DR

This paper introduces Tensor Product Generation Networks (TPGN), a deep learning architecture for NLP that encodes symbolic structures using tensor products, enabling interpretable representations and improved image-caption generation.

Contribution

The paper proposes TPGN, a novel neural network architecture that integrates Tensor Product Representations for better symbolic encoding and interpretability in NLP tasks.

Findings

TPGN outperforms LSTM baselines on COCO image-caption dataset.

Internal representations contain significant grammatical information.

The model's structure allows interpretation of generated sequences as grammatical plans.

Abstract

We present a new approach to the design of deep networks for natural language processing (NLP), based on the general technique of Tensor Product Representations (TPRs) for encoding and processing symbol structures in distributed neural networks. A network architecture --- the Tensor Product Generation Network (TPGN) --- is proposed which is capable in principle of carrying out TPR computation, but which uses unconstrained deep learning to design its internal representations. Instantiated in a model for image-caption generation, TPGN outperforms LSTM baselines when evaluated on the COCO dataset. The TPR-capable structure enables interpretation of internal representations and operations, which prove to contain considerable grammatical content. Our caption-generation model can be interpreted as generating sequences of grammatical categories and retrieving words by their categories from a plan encoded as a distributed representation.