NEU at WNUT-2020 Task 2: Data Augmentation To Tell BERT That Death Is Not Necessarily Informative
Kumud Chauhan

TL;DR
This paper develops a BERT-based classifier for identifying informative COVID-19 tweets and introduces a data augmentation method to improve its robustness against misleading signals like mentions of death counts.
Contribution
It reveals how simple signals can mislead BERT and proposes a data augmentation technique to enhance classifier robustness and generalization.
Findings
Adding death mentions drastically reduces BERT performance.
Simple patterns can degrade BERT's ability to identify informative tweets.
Data augmentation improves BERT's robustness and generalization.
Abstract
Millions of people around the world are sharing COVID-19 related information on social media platforms. Since not all the information shared on the social media is useful, a machine learning system to identify informative posts can help users in finding relevant information. In this paper, we present a BERT classifier system for W-NUT2020 Shared Task 2: Identification of Informative COVID-19 English Tweets. Further, we show that BERT exploits some easy signals to identify informative tweets, and adding simple patterns to uninformative tweets drastically degrades BERT performance. In particular, simply adding 10 deaths to tweets in dev set, reduces BERT F1- score from 92.63 to 7.28. We also propose a simple data augmentation technique that helps in improving the robustness and generalization ability of the BERT classifier.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsLinear Layer · Softmax · Layer Normalization · Dense Connections · Weight Decay · Dropout · Linear Warmup With Linear Decay · Attention Dropout · WordPiece · Multi-Head Attention
