aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model

TL;DR

This paper introduces aNMM, an attention-based neural model for ranking short answer texts that outperforms existing neural approaches and is competitive with feature-engineered methods, especially when combined with additional features.

Contribution

The paper proposes a novel attention-based neural matching model for short answer ranking that effectively incorporates question term importance and value-shared weighting, improving performance.

Findings

aNMM significantly outperforms other neural models on TREC QA data.

aNMM is competitive with feature-engineered models.

Combining aNMM with additional features yields the best results.

Abstract

As an alternative to question answering methods based on feature engineering, deep learning approaches such as convolutional neural networks (CNNs) and Long Short-Term Memory Models (LSTMs) have recently been proposed for semantic matching of questions and answers. To achieve good results, however, these models have been combined with additional features such as word overlap or BM25 scores. Without this combination, these models perform significantly worse than methods based on linguistic feature engineering. In this paper, we propose an attention based neural matching model for ranking short answer text. We adopt value-shared weighting scheme instead of position-shared weighting scheme for combining different matching signals and incorporate question term importance learning using question attention network. Using the popular benchmark TREC QA data, we show that the relatively simple aNMM model can significantly outperform other neural network models that have been used for the question answering task, and is competitive with models that are combined with additional features. When aNMM is combined with additional features, it outperforms all baselines.