MolMetaLM: a Physicochemical Knowledge-Guided Molecular Meta Language Model

TL;DR

MolMetaLM introduces a physicochemical knowledge-guided molecular language model that leverages a specialized meta language paradigm to improve understanding and generation of molecular properties and structures.

Contribution

The paper presents a novel meta language framework tailored for molecules, integrating physicochemical knowledge and generating diverse pretraining tasks for enhanced molecular modeling.

Findings

Achieves strong performance in property prediction and molecule generation

Effectively infers molecular conformations and optimizes molecules

Demonstrates the importance of physicochemical knowledge in language models

Abstract

Most current molecular language models transfer the masked language model or image-text generation model from natural language processing to molecular field. However, molecules are not solely characterized by atom/bond symbols; they encapsulate important physical/chemical properties. Moreover, normal language models bring grammar rules that are irrelevant for understanding molecules. In this study, we propose a novel physicochemical knowledge-guided molecular meta language framework MolMetaLM. We design a molecule-specialized meta language paradigm, formatted as multiple <S,P,O> (subject, predicate, object) knowledge triples sharing the same S (i.e., molecule) to enhance learning the semantic relationships between physicochemical knowledge and molecules. By introducing different molecular knowledge and noises, the meta language paradigm generates tens of thousands of pretraining tasks. By recovering the token/sequence/order-level noises, MolMetaLM exhibits proficiency in large-scale benchmark evaluations involving property prediction, molecule generation, conformation inference, and molecular optimization. Through MolMetaLM, we offer a new insight for designing language models.