COVID-19: Comparative Analysis of Methods for Identifying Articles Related to Therapeutics and Vaccines without Using Labeled Data

TL;DR

This paper compares six transfer-learning and unsupervised text classification methods for identifying COVID-19 related articles on therapeutics and vaccines without labeled data, highlighting strengths and limitations of each approach.

Contribution

It introduces an analysis framework based on task-specific terms and develops an improved unsupervised ensemble method for article screening.

Findings

BERT trained on search results performs well but misses abstracts lacking task-specific terms.

Unsupervised ensemble improves classification accuracy.

Task-specific term presence is a key factor in method performance.

Abstract

Here we proposed an approach to analyze text classification methods based on the presence or absence of task-specific terms (and their synonyms) in the text. We applied this approach to study six different transfer-learning and unsupervised methods for screening articles relevant to COVID-19 vaccines and therapeutics. The analysis revealed that while a BERT model trained on search-engine results generally performed well, it miss-classified relevant abstracts that did not contain task-specific terms. We used this insight to create a more effective unsupervised ensemble.