Leak Proof PDBBind: A Reorganized Dataset of Protein-Ligand Complexes for More Generalizable Binding Affinity Prediction

TL;DR

This study introduces a leak-proof version of the PDBBind dataset to evaluate protein-ligand scoring functions more reliably, demonstrating improved performance of retrained models on new complexes and proposing IGN as a top scorer.

Contribution

The paper provides a carefully curated, leak-proof dataset split for more accurate evaluation of scoring functions and retrains popular models to improve their generalizability on unseen data.

Findings

Retrained models perform better on leak-proof data.

IGN outperforms other scoring functions for new complexes.

The new dataset reduces data leakage issues in evaluation.

Abstract

The majority of machine learning scoring functions used in drug discovery for predicting protein-ligand binding poses and affinities have been trained on the PDBBind dataset. However, it is unclear whether these new scoring functions are actually an improvement over traditional models since often the training and test sets are cross-contaminated with proteins and ligands with high similarity, and hence they may not perform comparably well in binding prediction of unrelated protein-ligand complexes. In this work we have carefully prepared a new split of the PDBBind data set to control for data leakage, defined as proteins and ligands with high sequence and structural similarity. The resulting leak-proof (LP)-PDBBind data is used to retrain four popular SFs: AutoDock Vina, Random Forest (RF)-Score, InteractionGraphNet (IGN), and DeepDTA, to better test their capabilities when applied to new protein-ligand complexes. In particular we have formulated a new independent data set, BDB2020+, by matching high quality binding free energies from BindingDB with co-crystalized ligand-protein complexes from the PDB that have been deposited since 2020. Based on all the benchmark results, the retrained models using LP-PDBBind consistently perform better, with IGN especially being recommended for scoring and ranking applications for new protein-ligand systems.