Discovering key topics from short, real-world medical inquiries via natural language processing and unsupervised learning

TL;DR

This paper presents an unsupervised machine learning method using NLP to automatically identify meaningful, medically relevant topics from large volumes of unsolicited medical inquiries, aiding pharmaceutical insights.

Contribution

It introduces a novel NLP and unsupervised learning approach that discovers key medical inquiry topics without needing ontologies or annotations.

Findings

Discovered medically relevant topics validated by specialists

Method effectively handles large volumes of inquiries

Enhances understanding of customer medical concerns

Abstract

Millions of unsolicited medical inquiries are received by pharmaceutical companies every year. It has been hypothesized that these inquiries represent a treasure trove of information, potentially giving insight into matters regarding medicinal products and the associated medical treatments. However, due to the large volume and specialized nature of the inquiries, it is difficult to perform timely, recurrent, and comprehensive analyses. Here, we propose a machine learning approach based on natural language processing and unsupervised learning to automatically discover key topics in real-world medical inquiries from customers. This approach does not require ontologies nor annotations. The discovered topics are meaningful and medically relevant, as judged by medical information specialists, thus demonstrating that unsolicited medical inquiries are a source of valuable customer insights. Our work paves the way for the machine-learning-driven analysis of medical inquiries in the pharmaceutical industry, which ultimately aims at improving patient care.