BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations

TL;DR

BioT5 is a novel pre-training framework that enhances cross-modal biological data integration by utilizing chemical knowledge, natural language associations, and robust molecular representations to improve performance in bio-entity related tasks.

Contribution

It introduces BioT5, a comprehensive pre-training model that effectively combines structured and unstructured biological knowledge with natural language, addressing limitations of previous models.

Findings

BioT5 outperforms existing models on various biological tasks.

Utilizes SELFIES for 100% robust molecular representations.

Effectively distinguishes and leverages structured and unstructured knowledge.

Abstract

Recent advancements in biological research leverage the integration of molecules, proteins, and natural language to enhance drug discovery. However, current models exhibit several limitations, such as the generation of invalid molecular SMILES, underutilization of contextual information, and equal treatment of structured and unstructured knowledge. To address these issues, we propose $\mathbf{BioT5}$, a comprehensive pre-training framework that enriches cross-modal integration in biology with chemical knowledge and natural language associations. $\mathbf{BioT5}$ utilizes SELFIES for $100%$ robust molecular representations and extracts knowledge from the surrounding context of bio-entities in unstructured biological literature. Furthermore, $\mathbf{BioT5}$ distinguishes between structured and unstructured knowledge, leading to more effective utilization of information. After fine-tuning, BioT5 shows superior performance across a wide range of tasks, demonstrating its strong capability of capturing underlying relations and properties of bio-entities. Our code is available at $\href{https://github.com/QizhiPei/BioT5}{Github}$.