SLEDGE: A Simple Yet Effective Baseline for COVID-19 Scientific Knowledge Search

TL;DR

SLEDGE is a simple yet effective search baseline for COVID-19 literature, leveraging SciBERT for re-ranking articles and achieving top performance in the TREC-COVID challenge.

Contribution

The paper introduces SLEDGE, a neural re-ranking system using SciBERT trained on general answer data and adapted for COVID-19 literature search, setting a new baseline.

Findings

SLEDGE outperforms previous baselines on TREC-COVID with nDCG@10 of 0.6844.

Filtering by date and count signals are promising directions for future research.

Neural methods can effectively leverage transfer learning for specialized domains.

Abstract

With worldwide concerns surrounding the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), there is a rapidly growing body of literature on the virus. Clinicians, researchers, and policy-makers need a way to effectively search these articles. In this work, we present a search system called SLEDGE, which utilizes SciBERT to effectively re-rank articles. We train the model on a general-domain answer ranking dataset, and transfer the relevance signals to SARS-CoV-2 for evaluation. We observe SLEDGE's effectiveness as a strong baseline on the TREC-COVID challenge (topping the learderboard with an nDCG@10 of 0.6844). Insights provided by a detailed analysis provide some potential future directions to explore, including the importance of filtering by date and the potential of neural methods that rely more heavily on count signals. We release the code to facilitate future work on this critical task at https://github.com/Georgetown-IR-Lab/covid-neural-ir