ILPS at TREC 2017 Common Core Track

TL;DR

This paper describes the IlpsUvA team's participation in TREC 2017 Common Core, introducing two methods: BOIR for hyperparameter optimization and NVSM for unsupervised document ranking, achieving competitive and diverse results.

Contribution

The paper presents two novel retrieval methods, BOIR and NVSM, demonstrating effective hyperparameter tuning and diverse unsupervised document representations in TREC 2017.

Findings

BOIR effectively optimizes hyperparameters for competitive performance.

NVSM produces diverse rankings with many unique relevant documents.

NVSM is among the top unsupervised runs for diversity.

Abstract

The TREC 2017 Common Core Track aimed at gathering a diverse set of participating runs and building a new test collection using advanced pooling methods. In this paper, we describe the participation of the IlpsUvA team at the TREC 2017 Common Core Track. We submitted runs created using two methods to the track: (1) BOIR uses Bayesian optimization to automatically optimize retrieval model hyperparameters. (2) NVSM is a latent vector space model where representations of documents and query terms are learned from scratch in an unsupervised manner. We find that BOIR is able to optimize hyperparameters as to find a system that performs competitively amongst track participants. NVSM provides rankings that are diverse, as it was amongst the top automated unsupervised runs that provided the most unique relevant documents.