Modeling Ambiguity, Subjectivity, and Diverging Viewpoints in Opinion Question Answering Systems

TL;DR

This paper introduces a new approach to opinion-based question answering that models ambiguity, subjectivity, and personalization, supported by a large dataset of questions and answers from product reviews.

Contribution

It presents a novel dataset and methodology that incorporate multiple divergent answers and personalization factors into opinion question answering systems.

Findings

Explicit modeling of divergence improves answer quality.

Personalization enhances relevance of answers.

Provides a large-scale dataset for future research.

Abstract

Product review websites provide an incredible lens into the wide variety of opinions and experiences of different people, and play a critical role in helping users discover products that match their personal needs and preferences. To help address questions that can't easily be answered by reading others' reviews, some review websites also allow users to pose questions to the community via a question-answering (QA) system. As one would expect, just as opinions diverge among different reviewers, answers to such questions may also be subjective, opinionated, and divergent. This means that answering such questions automatically is quite different from traditional QA tasks, where it is assumed that a single `correct' answer is available. While recent work introduced the idea of question-answering using product reviews, it did not account for two aspects that we consider in this paper: (1) Questions have multiple, often divergent, answers, and this full spectrum of answers should somehow be used to train the system; and (2) What makes a `good' answer depends on the asker and the answerer, and these factors should be incorporated in order for the system to be more personalized. Here we build a new QA dataset with 800 thousand questions---and over 3.1 million answers---and show that explicitly accounting for personalization and ambiguity leads both to quantitatively better answers, but also a more nuanced view of the range of supporting, but subjective, opinions.