Addressing Complex and Subjective Product-Related Queries with Customer Reviews

TL;DR

This paper introduces Moqa, a machine learning framework that automatically identifies relevant reviews for complex, subjective product queries by learning from a large dataset of questions and reviews, improving online product information retrieval.

Contribution

The paper presents a novel mixture-of-experts model that effectively links product reviews to specific user queries, addressing the challenge of subjective and complex question answering in e-commerce.

Findings

Moqa outperforms baseline methods in relevance detection.

The system effectively handles both binary and open-ended queries.

Qualitative analysis shows human evaluators find the surfaced reviews relevant.

Abstract

Online reviews are often our first port of call when considering products and purchases online. When evaluating a potential purchase, we may have a specific query in mind, e.g. `will this baby seat fit in the overhead compartment of a 747?' or `will I like this album if I liked Taylor Swift's 1989?'. To answer such questions we must either wade through huge volumes of consumer reviews hoping to find one that is relevant, or otherwise pose our question directly to the community via a Q/A system. In this paper we hope to fuse these two paradigms: given a large volume of previously answered queries about products, we hope to automatically learn whether a review of a product is relevant to a given query. We formulate this as a machine learning problem using a mixture-of-experts-type framework---here each review is an `expert' that gets to vote on the response to a particular query; simultaneously we learn a relevance function such that `relevant' reviews are those that vote correctly. At test time this learned relevance function allows us to surface reviews that are relevant to new queries on-demand. We evaluate our system, Moqa, on a novel corpus of 1.4 million questions (and answers) and 13 million reviews. We show quantitatively that it is effective at addressing both binary and open-ended queries, and qualitatively that it surfaces reviews that human evaluators consider to be relevant.