Scalable Multi-Task Transfer Learning for Molecular Property Prediction

TL;DR

This paper introduces a scalable, data-driven bi-level optimization approach for multi-task transfer learning in molecular property prediction, overcoming manual design limitations and improving prediction accuracy and training efficiency.

Contribution

It presents a novel automated method for optimizing transfer ratios in multi-task learning, enhancing scalability and performance in molecular property prediction.

Findings

Improved prediction accuracy for 40 molecular properties.

Accelerated training convergence.

Automated transfer ratio optimization outperforms manual methods.

Abstract

Molecules have a number of distinct properties whose importance and application vary. Often, in reality, labels for some properties are hard to achieve despite their practical importance. A common solution to such data scarcity is to use models of good generalization with transfer learning. This involves domain experts for designing source and target tasks whose features are shared. However, this approach has limitations: i). Difficulty in accurate design of source-target task pairs due to the large number of tasks, and ii). corresponding computational burden verifying many trials and errors of transfer learning design, thereby iii). constraining the potential of foundation modeling of multi-task molecular property prediction. We address the limitations of the manual design of transfer learning via data-driven bi-level optimization. The proposed method enables scalable multi-task transfer learning for molecular property prediction by automatically obtaining the optimal transfer ratios. Empirically, the proposed method improved the prediction performance of 40 molecular properties and accelerated training convergence.