Multi-Task Fine-Tuning Enables Robust Out-of-Distribution Generalization in Atomistic Models

TL;DR

This paper introduces multi-task fine-tuning (MFT) for atomistic models, which enhances out-of-distribution generalization by preserving chemical priors during adaptation, outperforming standard methods across benchmarks.

Contribution

The paper proposes MFT, a novel fine-tuning approach that jointly optimizes property prediction and a force-field objective to improve OOD robustness in atomistic models.

Findings

MFT consistently improves OOD generalization across benchmarks.

Standard fine-tuning causes representation collapse degrading OOD performance.

MFT approaches the theoretical limit of in-distribution accuracy for OOD tasks.

Abstract

Accurate de novo molecular and materials design requires structure-property models that generalize beyond known regimes. Although pretrained atomistic models achieve strong in-distribution accuracy after fine-tuning, their reliability under out-of-distribution (OOD) conditions remains unclear. We identify a critical failure mode in downstream adaptation: standard fine-tuning induces representation collapse, erasing pretrained chemical and structural priors and severely degrading OOD performance. To address this limitation, we propose multi-task fine-tuning (MFT), which jointly optimizes downstream property prediction with a physically grounded force-field objective inherited from pretraining. This approach preserves essential chemical priors while enabling task-specific adaptation. Across molecular and materials benchmarks, MFT consistently improves OOD generalization, approaching the theoretical limit set by in-distribution accuracy, while outperforming standard fine-tuning, training from scratch, and state-of-the-art task-specific models. These results establish safe adaptation as a central requirement for large atomistic models and position MFT as a practical and data-efficient pathway toward robust molecular and materials discovery.