Mining of health and disease events on Twitter: validating search protocols within the setting of Indonesia

TL;DR

This study validates a Twitter search protocol for health-related terms in Indonesia, demonstrating its effectiveness in detecting disease-related conversations and suggesting Twitter's potential as a real-time health monitoring tool.

Contribution

The paper introduces a validated search protocol for health events on Twitter, using CART algorithm and accuracy metrics, to enhance real-time health surveillance in Indonesia.

Findings

High accuracy with AUC over 0.8 in detecting disease tweets

Twitter can serve as a real-time proxy for health event monitoring

Validated search protocol improves disease detection from social media data

Abstract

This study seeks to validate a search protocol of ill health-related terms using Twitter data which can later be used to understand if, and how, Twitter can reveal information on the current health situation. We extracted conversations related to health and disease postings on Twitter using a set of pre-defined keywords, assessed the prevalence, frequency, and timing of such content in these conversations, and validated how this search protocol was able to detect relevant disease tweets. Classification and Regression Trees (CART) algorithm was used to train and test search protocols of disease and health hits comparing to those identified by our team. The accuracy of predictions showed a good validity with AUC beyond 0.8. Our study shows that monitoring of public sentiment on Twitter can be used as a real-time proxy for health events.