Evaluating Embeddings for One-Shot Classification of Doctor-AI Consultations

TL;DR

This paper evaluates various text embeddings for classifying healthcare consultation texts, demonstrating that certain embeddings effectively capture semantic features and improve classification accuracy in low-data scenarios.

Contribution

It systematically compares multiple embeddings for one-shot classification of medical consultation texts, highlighting the effectiveness of Word2Vec, GloVe, and GPT2 embeddings.

Findings

Word2Vec, GloVe, and character n-grams perform well in classification.

GPT2 embeddings show notable performance in this task.

Machine learning models improve communication quality with limited training data.

Abstract

Effective communication between healthcare providers and patients is crucial to providing high-quality patient care. In this work, we investigate how Doctor-written and AI-generated texts in healthcare consultations can be classified using state-of-the-art embeddings and one-shot classification systems. By analyzing embeddings such as bag-of-words, character n-grams, Word2Vec, GloVe, fastText, and GPT2 embeddings, we examine how well our one-shot classification systems capture semantic information within medical consultations. Results show that the embeddings are capable of capturing semantic features from text in a reliable and adaptable manner. Overall, Word2Vec, GloVe and Character n-grams embeddings performed well, indicating their suitability for modeling targeted to this task. GPT2 embedding also shows notable performance, indicating its suitability for models tailored to this task as well. Our machine learning architectures significantly improved the quality of health conversations when training data are scarce, improving communication between patients and healthcare providers.