Monitoring Covid-19 on social media using a novel triage and diagnosis approach

TL;DR

This paper presents an NLP pipeline that extracts symptoms and related concepts from social media posts to triage and diagnose COVID-19, achieving high accuracy and providing insights into symptom importance.

Contribution

The study introduces a novel end-to-end NLP approach combining concept extraction and machine learning for COVID-19 triage and diagnosis from social media data.

Findings

F1 scores of 71-96% for triage and diagnosis

End-to-end models perform comparably to models trained on labeled data

Important diagnostic features are not always the most frequent symptoms

Abstract

Objective: This study aims to develop an end-to-end natural language processing pipeline for triage and diagnosis of COVID-19 from patient-authored social media posts, in order to provide researchers and public health practitioners with additional information on the symptoms, severity and prevalence of the disease rather than to provide an actionable decision at the individual level. Materials and Methods: The text processing pipeline first extracts COVID-19 symptoms and related concepts such as severity, duration, negations, and body parts from patients' posts using conditional random fields. An unsupervised rule-based algorithm is then applied to establish relations between concepts in the next step of the pipeline. The extracted concepts and relations are subsequently used to construct two different vector representations of each post. These vectors are applied separately to build support vector machine learning models to triage patients into three categories and diagnose them for COVID-19. Results: We report that macro- and micro-averaged F1 scores in the range of 71-96% and 61-87%, respectively, for the triage and diagnosis of COVID-19, when the models are trained on human labelled data. Our experimental results indicate that similar performance can be achieved when the models are trained using predicted labels from concept extraction and rule-based classifiers, thus yielding end-to-end machine learning. Also, we highlight important features uncovered by our diagnostic machine learning models and compare them with the most frequent symptoms revealed in another COVID-19 dataset. In particular, we found that the most important features are not always the most frequent ones.