Characterizing Question Facets for Complex Answer Retrieval

TL;DR

This paper introduces two innovative methods for complex answer retrieval that leverage question facet utility, significantly improving ranking performance on the TREC CAR benchmark.

Contribution

The paper presents novel approaches to incorporate and learn question facet utility within neural ranking models for complex answer retrieval.

Findings

Achieved top rank in the 2017 TREC CAR benchmark.

Up to 26% performance improvement over previous methods.

Demonstrated effectiveness of facet utility modeling in neural ranking.

Abstract

Complex answer retrieval (CAR) is the process of retrieving answers to questions that have multifaceted or nuanced answers. In this work, we present two novel approaches for CAR based on the observation that question facets can vary in utility: from structural (facets that can apply to many similar topics, such as 'History') to topical (facets that are specific to the question's topic, such as the 'Westward expansion' of the United States). We first explore a way to incorporate facet utility into ranking models during query term score combination. We then explore a general approach to reform the structure of ranking models to aid in learning of facet utility in the query-document term matching phase. When we use our techniques with a leading neural ranker on the TREC CAR dataset, our methods rank first in the 2017 TREC CAR benchmark, and yield up to 26% higher performance than the next best method.