Identifying Structures in Social Conversations in NSCLC Patients through the Semi-Automatic extraction of Topical Taxonomies

TL;DR

This paper presents a semi-automatic method for extracting topical taxonomies from social media conversations related to NSCLC patients, aiming to structure and analyze patient needs effectively.

Contribution

It introduces a novel approach for automatic taxonomy generation from social conversations, including label creation and optimal category determination, extending prior research.

Findings

Successfully applied to social media data on cancer patients

Generated meaningful topical taxonomies capturing latent knowledge

Enhanced understanding of patient needs from social conversations

Abstract

The exploration of social conversations for addressing patient's needs is an important analytical task in which many scholarly publications are contributing to fill the knowledge gap in this area. The main difficulty remains the inability to turn such contributions into pragmatic processes the pharmaceutical industry can leverage in order to generate insight from social media data, which can be considered as one of the most challenging source of information available today due to its sheer volume and noise. This study is based on the work by Scott Spangler and Jeffrey Kreulen and applies it to identify structure in social media through the extraction of a topical taxonomy able to capture the latent knowledge in social conversations in health-related sites. The mechanism for automatically identifying and generating a taxonomy from social conversations is developed and pressured tested using public data from media sites focused on the needs of cancer patients and their families. Moreover, a novel method for generating the category's label and the determination of an optimal number of categories is presented which extends Scott and Jeffrey's research in a meaningful way. We assume the reader is familiar with taxonomies, what they are and how they are used.