COVID-19 India Dataset: Parsing COVID-19 Data in Daily Health Bulletins from States in India

TL;DR

This paper presents an automated method for extracting COVID-19 data from Indian health bulletins, addressing accessibility issues caused by unstructured data and manual updates, using PDF parsing and machine learning.

Contribution

The paper introduces a novel automated approach combining PDF parsing and machine learning to extract structured COVID-19 data from Indian health bulletins.

Findings

Successfully extracted data from unstructured health bulletins

Generated a comprehensive COVID-19 dataset for India

Enabled easier access to detailed pandemic data

Abstract

While India has been one of the hotspots of COVID-19, data about the pandemic from the country has proved to be largely inaccessible at scale. Much of the data exists in unstructured form on the web, and limited aspects of such data are available through public APIs maintained manually through volunteer effort. This has proved to be difficult both in terms of ease of access to detailed data and with regards to the maintenance of manual data-keeping over time. This paper reports on our effort at automating the extraction of such data from public health bulletins with the help of a combination of classical PDF parsers and state-of-the-art machine learning techniques. In this paper, we will describe the automated data-extraction technique, the nature of the generated data, and exciting avenues of ongoing work.