Answering Subjective Induction Questions on Products by Summarizing Multi-sources Multi-viewpoints Knowledge

TL;DR

This paper introduces a new task of answering subjective product questions by summarizing multi-source opinions and facts, proposing a three-step method, and creating a large dataset for evaluation.

Contribution

It defines a novel subjective QA task, develops a multi-source summarization approach, and constructs the first large-scale dataset for this problem.

Findings

The proposed method effectively aggregates multi-source knowledge.

The dataset SupQA enables comprehensive evaluation of subjective product QA.

Experimental results demonstrate the approach's superiority over baselines.

Abstract

This paper proposes a new task in the field of Answering Subjective Induction Question on Products (SUBJPQA). The answer to this kind of question is non-unique, but can be interpreted from many perspectives. For example, the answer to 'whether the phone is heavy' has a variety of different viewpoints. A satisfied answer should be able to summarize these subjective opinions from multiple sources and provide objective knowledge, such as the weight of a phone. That is quite different from the traditional QA task, in which the answer to a factoid question is unique and can be found from a single data source. To address this new task, we propose a three-steps method. We first retrieve all answer-related clues from multiple knowledge sources on facts and opinions. The implicit commonsense facts are also collected to supplement the necessary but missing contexts. We then capture their relevance with the questions by interactive attention. Next, we design a reinforcement-based summarizer to aggregate all these knowledgeable clues. Based on a template-controlled decoder, we can output a comprehensive and multi-perspective answer. Due to the lack of a relevant evaluated benchmark set for the new task, we construct a large-scale dataset, named SupQA, consisting of 48,352 samples across 15 product domains. Evaluation results show the effectiveness of our approach.