Interpretable Self-supervised Multi-task Learning for COVID-19 Information Retrieval and Extraction

TL;DR

This paper introduces an interpretable self-supervised multi-task learning model designed for COVID-19 information retrieval and extraction, improving robustness, data efficiency, and generalization during the pandemic.

Contribution

It presents a novel multi-task, self-supervised approach that enhances COVID-19 related NLP tasks with interpretability and superior performance over baselines.

Findings

Outperforms baselines with 0.08 micro-f score improvement in IE.

Achieves 0.05 higher MAP in IR tasks.

Zero- and few-shot IE performances are significantly better than baselines.

Abstract

The rapidly evolving literature of COVID-19 related articles makes it challenging for NLP models to be effectively trained for information retrieval and extraction with the corresponding labeled data that follows the current distribution of the pandemic. On the other hand, due to the uncertainty of the situation, human experts' supervision would always be required to double check the decision making of these models highlighting the importance of interpretability. In the light of these challenges, this study proposes an interpretable self-supervised multi-task learning model to jointly and effectively tackle the tasks of information retrieval (IR) and extraction (IE) during the current emergency health crisis situation. Our results show that our model effectively leverage the multi-task and self-supervised learning to improve generalization, data efficiency and robustness to the ongoing dataset shift problem. Our model outperforms baselines in IE and IR tasks, respectively by micro-f score of 0.08 (LCA-F score of 0.05), and MAP of 0.05 on average. In IE the zero- and few-shot learning performances are on average 0.32 and 0.19 micro-f score higher than those of the baselines.