Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN

TL;DR

This paper introduces AR-SMCNN, a novel neural network model combining RNNs and CNNs with attention and similarity matrices to improve question answering over Freebase, capturing both semantic and literal word interactions.

Contribution

It proposes a new model that preserves original word interactions and hierarchical information, outperforming existing methods on the SimpleQuestion benchmark.

Findings

Outperforms state-of-the-art on SimpleQuestion benchmark

Effectively captures semantic and literal word interactions

Reduces noise in entity detection

Abstract

With the rapid growth of knowledge bases (KBs), question answering over knowledge base, a.k.a. KBQA has drawn huge attention in recent years. Most of the existing KBQA methods follow so called encoder-compare framework. They map the question and the KB facts to a common embedding space, in which the similarity between the question vector and the fact vectors can be conveniently computed. This, however, inevitably loses original words interaction information. To preserve more original information, we propose an attentive recurrent neural network with similarity matrix based convolutional neural network (AR-SMCNN) model, which is able to capture comprehensive hierarchical information utilizing the advantages of both RNN and CNN. We use RNN to capture semantic-level correlation by its sequential modeling nature, and use an attention mechanism to keep track of the entities and relations simultaneously. Meanwhile, we use a similarity matrix based CNN with two-directions pooling to extract literal-level words interaction matching utilizing CNNs strength of modeling spatial correlation among data. Moreover, we have developed a new heuristic extension method for entity detection, which significantly decreases the effect of noise. Our method has outperformed the state-of-the-arts on SimpleQuestion benchmark in both accuracy and efficiency.