MMPolymer: A Multimodal Multitask Pretraining Framework for Polymer Property Prediction

TL;DR

MMPolymer is a novel framework that integrates 1D and 3D polymer structural data through multimodal pretraining, significantly improving polymer property prediction accuracy and robustness.

Contribution

This work introduces MMPolymer, the first multimodal multitask pretraining framework for polymers that effectively combines 1D sequences and 3D structures, including a new 'Star Substitution' strategy for limited 3D data.

Findings

Achieves state-of-the-art results in polymer property prediction.

Single modality fine-tuning with MMPolymer outperforms existing methods.

Effectively leverages limited 3D structural data for improved predictions.

Abstract

Polymers are high-molecular-weight compounds constructed by the covalent bonding of numerous identical or similar monomers so that their 3D structures are complex yet exhibit unignorable regularity. Typically, the properties of a polymer, such as plasticity, conductivity, bio-compatibility, and so on, are highly correlated with its 3D structure. However, existing polymer property prediction methods heavily rely on the information learned from polymer SMILES sequences (P-SMILES strings) while ignoring crucial 3D structural information, resulting in sub-optimal performance. In this work, we propose MMPolymer, a novel multimodal multitask pretraining framework incorporating polymer 1D sequential and 3D structural information to encourage downstream polymer property prediction tasks. Besides, considering the scarcity of polymer 3D data, we further introduce the "Star Substitution" strategy to extract 3D structural information effectively. During pretraining, in addition to predicting masked tokens and recovering clear 3D coordinates, MMPolymer achieves the cross-modal alignment of latent representations. Then we further fine-tune the pretrained MMPolymer for downstream polymer property prediction tasks in the supervised learning paradigm. Experiments show that MMPolymer achieves state-of-the-art performance in downstream property prediction tasks. Moreover, given the pretrained MMPolymer, utilizing merely a single modality in the fine-tuning phase can also outperform existing methods, showcasing the exceptional capability of MMPolymer in polymer feature extraction and utilization.