Generalization Beyond Benchmarks: Evaluating Learnable Protein-Ligand Scoring Functions on Unseen Targets

TL;DR

This paper assesses how well current machine learning-based protein-ligand scoring functions perform on unseen targets, highlighting the limitations of existing benchmarks and exploring pretraining and simple methods to improve generalization.

Contribution

It critically evaluates the generalization of scoring functions beyond standard benchmarks and proposes more rigorous evaluation protocols and potential improvements.

Findings

Benchmarks do not reflect true generalization challenges.

Large-scale pretraining shows potential to improve predictions.

Simple test-target data methods can enhance scoring accuracy.

Abstract

As machine learning becomes increasingly central to molecular design, it is vital to ensure the reliability of learnable protein-ligand scoring functions on novel protein targets. While many scoring functions perform well on standard benchmarks, their ability to generalize beyond training data remains a significant challenge. In this work, we evaluate the generalization capability of state-of-the-art scoring functions on dataset splits that simulate evaluation on targets with a limited number of known structures and experimental affinity measurements. Our analysis reveals that the commonly used benchmarks do not reflect the true challenge of generalizing to novel targets. We also investigate whether large-scale self-supervised pretraining can bridge this generalization gap and we provide preliminary evidence of its potential. Furthermore, we probe the efficacy of simple methods that leverage limited test-target data to improve scoring function performance. Our findings underscore the need for more rigorous evaluation protocols and offer practical guidance for designing scoring functions with predictive power extending to novel protein targets.