Assessing the Benefits of Model Ensembles in Neural Re-Ranking for Passage Retrieval

TL;DR

This paper experimentally evaluates how model ensembling improves neural passage reranking, demonstrating that ensembling enhances ranking quality especially when combined with supervised learning-to-rank methods.

Contribution

It introduces the application of Fast Geometric Ensembling to neural reranking models and compares various ensembling techniques for passage retrieval.

Findings

Ensembling improves ranking quality in neural passage reranking.

Supervised learning-to-rank benefits most from ensembling.

Unsupervised rank aggregation also shows improvements.

Abstract

Our work aimed at experimentally assessing the benefits of model ensembling within the context of neural methods for passage reranking. Starting from relatively standard neural models, we use a previous technique named Fast Geometric Ensembling to generate multiple model instances from particular training schedules, then focusing or attention on different types of approaches for combining the results from the multiple model instances (e.g., averaging the ranking scores, using fusion methods from the IR literature, or using supervised learning-to-rank). Tests with the MS-MARCO dataset show that model ensembling can indeed benefit the ranking quality, particularly with supervised learning-to-rank although also with unsupervised rank aggregation.