Transferring a molecular foundation model for polymer property predictions

TL;DR

This paper demonstrates that transfer learning from transformer models pretrained on small molecules can effectively predict polymer properties, reducing the need for costly data augmentation in polymer science.

Contribution

It introduces a transfer learning approach using pretrained small molecule transformers to improve polymer property predictions, offering a data-efficient alternative.

Findings

Pretrained models on small molecules perform comparably to augmented polymer datasets.

Transfer learning reduces computational costs in polymer property prediction.

The approach achieves high accuracy on benchmark tasks.

Abstract

Transformer-based large language models have remarkable potential to accelerate design optimization for applications such as drug development and materials discovery. Self-supervised pretraining of transformer models requires large-scale datasets, which are often sparsely populated in topical areas such as polymer science. State-of-the-art approaches for polymers conduct data augmentation to generate additional samples but unavoidably incurs extra computational costs. In contrast, large-scale open-source datasets are available for small molecules and provide a potential solution to data scarcity through transfer learning. In this work, we show that using transformers pretrained on small molecules and fine-tuned on polymer properties achieve comparable accuracy to those trained on augmented polymer datasets for a series of benchmark prediction tasks.