Bio+Clinical BERT, BERT Base, and CNN Performance Comparison for Predicting Drug-Review Satisfaction

TL;DR

This study compares NLP models including Bio+Clinical BERT, BERT Base, and CNN for classifying patient drug review satisfaction, highlighting Bio+Clinical BERT's superior performance in medical text analysis.

Contribution

It introduces a comparative analysis of domain-specific BERT and CNN models for drug review sentiment classification, demonstrating Bio+Clinical BERT's effectiveness.

Findings

Bio+Clinical BERT outperforms BERT Base with 11% higher macro F1 score.

Bio+Clinical BERT is especially effective with medical jargon.

CNN effectively identifies key words for sentiment classification.

Abstract

The objective of this study is to develop natural language processing (NLP) models that can analyze patients' drug reviews and accurately classify their satisfaction levels as positive, neutral, or negative. Such models would reduce the workload of healthcare professionals and provide greater insight into patients' quality of life, which is a critical indicator of treatment effectiveness. To achieve this, we implemented and evaluated several classification models, including a BERT base model, Bio+Clinical BERT, and a simpler CNN. Results indicate that the medical domain-specific Bio+Clinical BERT model significantly outperformed the general domain base BERT model, achieving macro f1 and recall score improvement of 11%, as shown in Table 2. Future research could explore how to capitalize on the specific strengths of each model. Bio+Clinical BERT excels in overall performance, particularly with medical jargon, while the simpler CNN demonstrates the ability to identify crucial words and accurately classify sentiment in texts with conflicting sentiments.