Gated Group Self-Attention for Answer Selection

TL;DR

This paper introduces Gated Group Self-Attention (GGSA), a novel deep learning model that improves answer selection in question answering systems by effectively capturing long-range dependencies with lower computational costs.

Contribution

The paper proposes GGSA, a new self-attention based model with an interaction mechanism, achieving state-of-the-art results in answer selection tasks.

Findings

GGSA outperforms existing models on two QA datasets.

GGSA achieves higher accuracy than global self-attention with lower computation cost.

GGSA effectively captures long-range dependencies in answer selection.

Abstract

Answer selection (answer ranking) is one of the key steps in many kinds of question answering (QA) applications, where deep models have achieved state-of-the-art performance. Among these deep models, recurrent neural network (RNN) based models are most popular, typically with better performance than convolutional neural network (CNN) based models. Nevertheless, it is difficult for RNN based models to capture the information about long-range dependency among words in the sentences of questions and answers. In this paper, we propose a new deep model, called gated group self-attention (GGSA), for answer selection. GGSA is inspired by global self-attention which is originally proposed for machine translation and has not been explored in answer selection. GGSA tackles the problem of global self-attention that local and global information cannot be well distinguished. Furthermore, an interaction mechanism between questions and answers is also proposed to enhance GGSA by a residual structure. Experimental results on two popular QA datasets show that GGSA can outperform existing answer selection models to achieve state-of-the-art performance. Furthermore, GGSA can also achieve higher accuracy than global self-attention for the answer selection task, with a lower computation cost.