What Are People Asking About COVID-19? A Question Classification Dataset

TL;DR

This paper introduces COVID-Q, a dataset of 1,690 COVID-19 questions categorized into 15 types, highlighting gaps in existing FAQs and providing baselines for question classification and clustering tasks.

Contribution

The paper presents COVID-Q, a new annotated dataset of COVID-19 questions with categories and clusters, and evaluates baseline models for question classification and clustering.

Findings

Most common questions are about transmission, prevention, and societal effects.

Many questions are unanswered by reputable health organization FAQs.

Baseline models achieve around 58% accuracy in classification and 50% in clustering.

Abstract

We present COVID-Q, a set of 1,690 questions about COVID-19 from 13 sources, which we annotate into 15 question categories and 207 question clusters. The most common questions in our dataset asked about transmission, prevention, and societal effects of COVID, and we found that many questions that appeared in multiple sources were not answered by any FAQ websites of reputable organizations such as the CDC and FDA. We post our dataset publicly at https://github.com/JerryWeiAI/COVID-Q. For classifying questions into 15 categories, a BERT baseline scored 58.1% accuracy when trained on 20 examples per category, and for a question clustering task, a BERT + triplet loss baseline achieved 49.5% accuracy. We hope COVID-Q can help either for direct use in developing applied systems or as a domain-specific resource for model evaluation.