Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization

TL;DR

This paper introduces MolPeg, a novel source-free data pruning framework for molecular tasks that improves generalization and efficiency by selectively removing less informative data using pretrained models.

Contribution

MolPeg is a new data pruning method that operates without source data, leveraging dual-model loss discrepancy to enhance molecular transfer learning performance.

Findings

Outperforms existing data pruning methods across four molecular tasks.

Can surpass full-dataset training performance with 60-70% data pruning.

Enhances both efficiency and generalization in molecular transfer learning.

Abstract

With the emergence of various molecular tasks and massive datasets, how to perform efficient training has become an urgent yet under-explored issue in the area. Data pruning (DP), as an oft-stated approach to saving training burdens, filters out less influential samples to form a coreset for training. However, the increasing reliance on pretrained models for molecular tasks renders traditional in-domain DP methods incompatible. Therefore, we propose a Molecular data Pruning framework for enhanced Generalization (MolPeg), which focuses on the source-free data pruning scenario, where data pruning is applied with pretrained models. By maintaining two models with different updating paces during training, we introduce a novel scoring function to measure the informativeness of samples based on the loss discrepancy. As a plug-and-play framework, MolPeg realizes the perception of both source and target domain and consistently outperforms existing DP methods across four downstream tasks. Remarkably, it can surpass the performance obtained from full-dataset training, even when pruning up to 60-70% of the data on HIV and PCBA dataset. Our work suggests that the discovery of effective data-pruning metrics could provide a viable path to both enhanced efficiency and superior generalization in transfer learning.