CROWN: Conversational Passage Ranking by Reasoning over Word Networks

TL;DR

CROWN is an unsupervised method for conversational passage ranking that leverages word networks to improve understanding of context and semantic similarity, enhancing retrieval accuracy in conversational systems.

Contribution

It introduces a novel word-proximity network approach for passage ranking that combines similarity and coherence without supervised training.

Findings

Improved nDCG scores over baseline on CAsT data

Achieved above-average AP@5 and nDCG@1000 scores

Demonstrated effectiveness of word networks in conversational retrieval

Abstract

Information needs around a topic cannot be satisfied in a single turn; users typically ask follow-up questions referring to the same theme and a system must be capable of understanding the conversational context of a request to retrieve correct answers. In this paper, we present our submission to the TREC Conversational Assistance Track 2019, in which such a conversational setting is explored. We propose a simple unsupervised method for conversational passage ranking by formulating the passage score for a query as a combination of similarity and coherence. To be specific, passages are preferred that contain words semantically similar to the words used in the question, and where such words appear close by. We built a word-proximity network (WPN) from a large corpus, where words are nodes and there is an edge between two nodes if they co-occur in the same passages in a statistically significant way, within a context window. Our approach, named CROWN, improved nDCG scores over a provided Indri baseline on the CAsT training data. On the evaluation data for CAsT, our best run submission achieved above-average performance with respect to AP@5 and nDCG@1000.