A Comparison of Word Embeddings for the Biomedical Natural Language Processing

TL;DR

This paper empirically evaluates various biomedical word embeddings trained on different textual resources, revealing their strengths and limitations for biomedical NLP tasks through qualitative and quantitative analyses.

Contribution

It provides a comprehensive comparison of word embeddings from clinical notes, biomedical publications, Wikipedia, and news, highlighting their relative effectiveness for biomedical NLP.

Findings

Embeddings from clinical notes and biomedical publications better capture medical semantics.

No single embedding type consistently outperforms others across all tasks.

Adding embeddings as features generally improves downstream biomedical NLP performance.

Abstract

Word embeddings have been widely used in biomedical Natural Language Processing (NLP) applications as they provide vector representations of words capturing the semantic properties of words and the linguistic relationship between words. Many biomedical applications use different textual resources (e.g., Wikipedia and biomedical articles) to train word embeddings and apply these word embeddings to downstream biomedical applications. However, there has been little work on evaluating the word embeddings trained from these resources.In this study, we provide an empirical evaluation of word embeddings trained from four different resources, namely clinical notes, biomedical publications, Wikipedia, and news. We performed the evaluation qualitatively and quantitatively. For the qualitative evaluation, we manually inspected five most similar medical words to a given set of target medical words, and then analyzed word embeddings through the visualization of those word embeddings. For the quantitative evaluation, we conducted both intrinsic and extrinsic evaluation. Based on the evaluation results, we can draw the following conclusions. First, the word embeddings trained on clinical notes and biomedical publications can capture the semantics of medical terms better, and find more relevant similar medical terms, and are closer to human experts' judgments, compared to these trained on Wikipedia and news. Second, there does not exist a consistent global ranking of word embedding quality for downstream biomedical NLP applications. However, adding word embeddings as extra features will improve results on most downstream tasks. Finally, the word embeddings trained on biomedical domain corpora do not necessarily have better performance than those trained on other general domain corpora for any downstream biomedical NLP tasks.