Automatic Question-Answering Using A Deep Similarity Neural Network

TL;DR

This paper introduces a deep learning model that embeds questions and answers to compute similarity scores, enabling automatic question-answering with high accuracy, adaptable to different datasets.

Contribution

It presents a novel deep similarity neural network for question-answering, combining neural probabilistic embeddings with transfer learning for improved performance.

Findings

Achieved high accuracy on public question-answering datasets.

Successfully transferred the model to customer-care chat data.

Demonstrated effectiveness of deep similarity neural networks in QA tasks.

Abstract

Automatic question-answering is a classical problem in natural language processing, which aims at designing systems that can automatically answer a question, in the same way as human does. In this work, we propose a deep learning based model for automatic question-answering. First the questions and answers are embedded using neural probabilistic modeling. Then a deep similarity neural network is trained to find the similarity score of a pair of answer and question. Then for each question, the best answer is found as the one with the highest similarity score. We first train this model on a large-scale public question-answering database, and then fine-tune it to transfer to the customer-care chat data. We have also tested our framework on a public question-answering database and achieved very good performance.