Answer Generation for Questions With Multiple Information Sources in E-Commerce

TL;DR

This paper introduces MSQAP, a novel pipeline for automatic answer generation in e-commerce that effectively combines multiple information sources, addressing relevance and sentiment ambiguity, and significantly improves answer accuracy and content quality.

Contribution

It presents the first integrated approach in e-commerce that combines reviews, questions, and specifications for natural language answer generation, with novel relevancy and ambiguity prediction modules.

Findings

Relevancy prediction model (BERT-QA) outperforms baselines with 12.36% F1 improvement.

Generation model (T5-QA) shows 35.02% ROUGE and 198.75% BLEU improvements.

Overall pipeline improves answer accuracy by 30.7% over standalone generation models.

Abstract

Automatic question answering is an important yet challenging task in E-commerce given the millions of questions posted by users about the product that they are interested in purchasing. Hence, there is a great demand for automatic answer generation systems that provide quick responses using related information about the product. There are three sources of knowledge available for answering a user posted query, they are reviews, duplicate or similar questions, and specifications. Effectively utilizing these information sources will greatly aid us in answering complex questions. However, there are two main challenges present in exploiting these sources: (i) The presence of irrelevant information and (ii) the presence of ambiguity of sentiment present in reviews and similar questions. Through this work we propose a novel pipeline (MSQAP) that utilizes the rich information present in the aforementioned sources by separately performing relevancy and ambiguity prediction before generating a response. Experimental results show that our relevancy prediction model (BERT-QA) outperforms all other variants and has an improvement of 12.36% in F1 score compared to the BERT-base baseline. Our generation model (T5-QA) outperforms the baselines in all content preservation metrics such as BLEU, ROUGE and has an average improvement of 35.02% in ROUGE and 198.75% in BLEU compared to the highest performing baseline (HSSC-q). Human evaluation of our pipeline shows us that our method has an overall improvement in accuracy of 30.7% over the generation model (T5-QA), resulting in our full pipeline-based approach (MSQAP) providing more accurate answers. To the best of our knowledge, this is the first work in the e-commerce domain that automatically generates natural language answers combining the information present in diverse sources such as specifications, similar questions, and reviews data.