Not-NUTs at W-NUT 2020 Task 2: A BERT-based System in Identifying Informative COVID-19 English Tweets

TL;DR

This paper presents a BERTweet-based ensemble system for classifying COVID-19 related tweets as informative or not, achieving near top performance in the W-NUT 2020 Task 2 competition.

Contribution

It introduces an ensemble approach using multiple BERTweet configurations for COVID-19 tweet informativeness classification, with additional experiments for better generalization.

Findings

Achieved F1 score close to top teams, within 1%.

Ensemble of BERTweet models improves classification accuracy.

Explored methods to enhance model generalization to new datasets.

Abstract

As of 2020 when the COVID-19 pandemic is full-blown on a global scale, people's need to have access to legitimate information regarding COVID-19 is more urgent than ever, especially via online media where the abundance of irrelevant information overshadows the more informative ones. In response to such, we proposed a model that, given an English tweet, automatically identifies whether that tweet bears informative content regarding COVID-19 or not. By ensembling different BERTweet model configurations, we have achieved competitive results that are only shy of those by top performing teams by roughly 1% in terms of F1 score on the informative class. In the post-competition period, we have also experimented with various other approaches that potentially boost generalization to a new dataset.