Analyzing Cancer Patients' Experiences with Embedding-based Topic Modeling and LLMs

TL;DR

This paper evaluates neural topic modeling and LLMs for extracting meaningful themes from cancer patient stories, demonstrating that domain-specific embeddings improve interpretability and relevance of identified topics.

Contribution

It introduces a comparative analysis of BERTopic and Top2Vec for patient interview summarization and demonstrates the effectiveness of clinical embeddings like BioClinicalBERT in healthcare-related topic modeling.

Findings

BERTopic outperforms Top2Vec in keyword extraction

BioClinicalBERT embeddings enhance topic interpretability

Identified dominant themes include care coordination and patient decision-making

Abstract

This study investigates the use of neural topic modeling and LLMs to uncover meaningful themes from patient storytelling data, to offer insights that could contribute to more patient-oriented healthcare practices. We analyze a collection of transcribed interviews with cancer patients (132,722 words in 13 interviews). We first evaluate BERTopic and Top2Vec for individual interview summarization by using similar preprocessing, chunking, and clustering configurations to ensure a fair comparison on Keyword Extraction. LLMs (GPT4) are then used for the next step topic labeling. Their outputs for a single interview (I0) are rated through a small-scale human evaluation, focusing on {coherence}, {clarity}, and {relevance}. Based on the preliminary results and evaluation, BERTopic shows stronger performance and is selected for further experimentation using three {clinically oriented embedding} models. We then analyzed the full interview collection with the best model setting. Results show that domain-specific embeddings improved topic \textit{precision} and \textit{interpretability}, with BioClinicalBERT producing the most consistent results across transcripts. The global analysis of the full dataset of 13 interviews, using the BioClinicalBERT embedding model, reveals the most dominant topics throughout all 13 interviews, namely ``Coordination and Communication in Cancer Care Management" and ``Patient Decision-Making in Cancer Treatment Journey''. Although the interviews are machine translations from Dutch to English, and clinical professionals are not involved in this evaluation, the findings suggest that neural topic modeling, particularly BERTopic, can help provide useful feedback to clinicians from patient interviews. This pipeline could support more efficient document navigation and strengthen the role of patients' voices in healthcare workflows.