Groupwise Query Performance Prediction with BERT

TL;DR

This paper introduces a novel BERT-based groupwise query performance prediction model that jointly models multiple queries to improve prediction accuracy, demonstrating effectiveness on standard TREC collections.

Contribution

It proposes the first groupwise BERT-based QPP model that leverages cross-query context for better performance prediction in IR tasks.

Findings

Outperforms existing pointwise models on TREC datasets

Effectively models query groups to enhance prediction accuracy

Code implementation is publicly available

Abstract

While large-scale pre-trained language models like BERT have advanced the state-of-the-art in IR, its application in query performance prediction (QPP) is so far based on pointwise modeling of individual queries. Meanwhile, recent studies suggest that the cross-attention modeling of a group of documents can effectively boost performances for both learning-to-rank algorithms and BERT-based re-ranking. To this end, a BERT-based groupwise QPP model is proposed, in which the ranking contexts of a list of queries are jointly modeled to predict the relative performance of individual queries. Extensive experiments on three standard TREC collections showcase effectiveness of our approach. Our code is available at https://github.com/VerdureChen/Group-QPP.