Finding the Answers with Definition Models

TL;DR

This paper enhances a neural definition model for answering crossword questions by applying bidirectional LSTMs, averaging states, expanding training data, and using sub-word units, leading to improved performance over previous models.

Contribution

It introduces specific extensions to the neural definition model, including bidirectional LSTMs and sub-word segmentation, to improve crossword question answering accuracy.

Findings

Extensions improve model performance on crossword questions.

Increased training data enhances results.

Sub-word segmentation benefits model accuracy.

Abstract

Inspired by a previous attempt to answer crossword questions using neural networks (Hill, Cho, Korhonen, & Bengio, 2015), this dissertation implements extensions to improve the performance of this existing definition model on the task of answering crossword questions. A discussion and evaluation of the original implementation finds that there are some ways in which the recurrent neural model could be extended. Insights from related fields neural language modeling and neural machine translation provide the justification and means required for these extensions. Two extensions are applied to the LSTM encoder, first taking the average of LSTM states across the sequence and secondly using a bidirectional LSTM, both implementations serve to improve model performance on a definitions and crossword test set. In order to improve performance on crossword questions, the training data is increased to include crossword questions and answers, and this serves to improve results on definitions as well as crossword questions. The final experiments are conducted using sub-word unit segmentation, first on the source side and then later preliminary experimentation is conducted to facilitate character-level output. Initially, an exact reproduction of the baseline results proves unsuccessful. Despite this, the extensions improve performance, allowing the definition model to surpass the performance of the recurrent neural network variants of the previous work (Hill, et al., 2015).