Ensemble approach for natural language question answering problem

TL;DR

This paper presents an ensemble neural network approach for natural language question answering, analyzing various models on the SQUAD dataset and demonstrating improved performance over existing models.

Contribution

It introduces a novel ensemble method combining multiple neural attention-based models for question answering, with detailed analysis on query subsets.

Findings

Ensemble model outperforms the Mnemonic Reader on SQUAD.

Analysis of model performance on specific query groups.

Study of neural attention mechanisms in question answering.

Abstract

Machine comprehension, answering a question depending on a given context paragraph is a typical task of Natural Language Understanding. It requires to model complex dependencies existing between the question and the context paragraph. There are many neural network models attempting to solve the problem of question answering. The best models have been selected, studied and compared with each other. All the selected models are based on the neural attention mechanism concept. Additionally, studies on a SQUAD dataset were performed. The subsets of queries were extracted and then each model was analyzed how it deals with specific group of queries. Based on these three model ensemble model was created and tested on SQUAD dataset. It outperforms the best Mnemonic Reader model.