University of Copenhagen Participation in TREC Health Misinformation Track 2020

TL;DR

This paper details the participation of the University of Copenhagen in the 2020 TREC Health Misinformation Track, employing a multi-step approach combining relevance, credibility, and misinformation scoring to improve document ranking.

Contribution

The paper introduces a novel multi-step ranking method that integrates relevance, credibility, and misinformation scores using machine learning and fusion techniques.

Findings

Effective combination of relevance, credibility, and misinformation scores improves ranking.

Use of Transformer-based stance detection enhances misinformation scoring.

Multiple merging strategies demonstrate robustness in document re-ranking.

Abstract

In this paper, we describe our participation in the TREC Health Misinformation Track 2020. We submitted $11$ runs to the Total Recall Task and 13 runs to the Ad Hoc task. Our approach consists of 3 steps: (1) we create an initial run with BM25 and RM3; (2) we estimate credibility and misinformation scores for the documents in the initial run; (3) we merge the relevance, credibility and misinformation scores to re-rank documents in the initial run. To estimate credibility scores, we implement a classifier which exploits features based on the content and the popularity of a document. To compute the misinformation score, we apply a stance detection approach with a pretrained Transformer language model. Finally, we use different approaches to merge scores: weighted average, the distance among score vectors and rank fusion.