Scalable Community Extraction of Text Networks for Automated Grouping in Medical Databases

TL;DR

This paper introduces a scalable community extraction algorithm tailored for large text networks in medical databases, improving group detection accuracy over manual tagging and aiding in patient safety analysis.

Contribution

It adapts a known community extraction method to large-scale text networks, enabling more accurate and scalable grouping in medical databases.

Findings

Generated groups are more accurate than manual tags.

Method is scalable to large text datasets.

Improves patient safety data analysis.

Abstract

Networks are ubiquitous in today's world. Community structure is a well-known feature of many empirical networks, and a lot of statistical methods have been developed for community detection. In this paper, we consider the problem of community structure in text networks,which is greatly relevant in medical errors and patient safety databases. We adapt a well-known community extraction method to develop a scalable algorithm for community extraction in large text databases. The application of our method on a real-world patient safety report database demonstrates that the groups generated from community extraction are much more accurate than manual tagging by frontline workers.