All You Need Is Synthetic Task Augmentation

TL;DR

This paper introduces a synthetic task augmentation method that jointly trains a Graph Transformer on real and synthetic tasks, significantly improving molecular property prediction performance without pretraining.

Contribution

The study presents a novel approach of using synthetic tasks derived from XGBoost models to enhance neural network training in molecular property prediction.

Findings

Consistent performance improvements across 19 tasks.

Outperforms single-task XGBoost in 16 of 19 cases.

Effective without feature injection or extensive pretraining.

Abstract

Injecting rule-based models like Random Forests into differentiable neural network frameworks remains an open challenge in machine learning. Recent advancements have demonstrated that pretrained models can generate efficient molecular embeddings. However, these approaches often require extensive pretraining and additional techniques, such as incorporating posterior probabilities, to boost performance. In our study, we propose a novel strategy that jointly trains a single Graph Transformer neural network on both sparse multitask molecular property experimental targets and synthetic targets derived from XGBoost models trained on Osmordred molecular descriptors. These synthetic tasks serve as independent auxiliary tasks. Our results show consistent and significant performance improvement across all 19 molecular property prediction tasks. For 16 out of 19 targets, the multitask Graph Transformer outperforms the XGBoost single-task learner. This demonstrates that synthetic task augmentation is an effective method for enhancing neural model performance in multitask molecular property prediction without the need for feature injection or pretraining.