Predicting drug-gene relations via analogy tasks with word embeddings

TL;DR

This paper shows that word embeddings trained on biomedical texts can predict drug-gene relations through analogy tasks, achieving performance comparable to large language models like GPT-4.

Contribution

It demonstrates that simple analogy computations on biomedical word embeddings can effectively predict drug-gene relations, including future unknown relations.

Findings

Embeddings encode drug-gene relation information.

Analogy tasks predict target genes from drugs.

Performance rivals GPT-4 in relation prediction.

Abstract

Natural language processing (NLP) is utilized in a wide range of fields, where words in text are typically transformed into feature vectors called embeddings. BioConceptVec is a specific example of embeddings tailored for biology, trained on approximately 30 million PubMed abstracts using models such as skip-gram. Generally, word embeddings are known to solve analogy tasks through simple vector arithmetic. For example, subtracting the vector for man from that of king and then adding the vector for woman yields a point that lies closer to queen in the embedding space. In this study, we demonstrate that BioConceptVec embeddings, along with our own embeddings trained on PubMed abstracts, contain information about drug-gene relations and can predict target genes from a given drug through analogy computations. We also show that categorizing drugs and genes using biological pathways improves performance. Furthermore, we illustrate that vectors derived from known relations in the past can predict unknown future relations in datasets divided by year. Despite the simplicity of implementing analogy tasks as vector additions, our approach demonstrated performance comparable to that of large language models such as GPT-4 in predicting drug-gene relations.