A Self-supervised Approach for Semantic Indexing in the Context of COVID-19 Pandemic

TL;DR

This paper introduces a self-supervised transformer-based model for automatic semantic indexing of COVID-19 literature, outperforming existing models and effectively handling rapidly evolving pandemic-related concepts.

Contribution

The study presents a novel self-supervised semantic indexing approach tailored for pandemic crises, demonstrating superior performance on COVID-19 literature compared to prior models.

Findings

Outperforms BioASQ Task 8a models by micro-F1 of 0.1

Achieves higher LCA-F score by 0.08 on average

Effective in detecting pandemic-specific supplementary concepts

Abstract

The pandemic has accelerated the pace at which COVID-19 scientific papers are published. In addition, the process of manually assigning semantic indexes to these papers by experts is even more time-consuming and overwhelming in the current health crisis. Therefore, there is an urgent need for automatic semantic indexing models which can effectively scale-up to newly introduced concepts and rapidly evolving distributions of the hyperfocused related literature. In this research, we present a novel semantic indexing approach based on the state-of-the-art self-supervised representation learning and transformer encoding exclusively suitable for pandemic crises. We present a case study on a novel dataset that is based on COVID-19 papers published and manually indexed in PubMed. Our study shows that our self-supervised model outperforms the best performing models of BioASQ Task 8a by micro-F1 score of 0.1 and LCA-F score of 0.08 on average. Our model also shows superior performance on detecting the supplementary concepts which is quite important when the focus of the literature has drastically shifted towards specific concepts related to the pandemic. Our study sheds light on the main challenges confronting semantic indexing models during a pandemic, namely new domains and drastic changes of their distributions, and as a superior alternative for such situations, propose a model founded on approaches which have shown auspicious performance in improving generalization and data efficiency in various NLP tasks. We also show the joint indexing of major Medical Subject Headings (MeSH) and supplementary concepts improves the overall performance.