Critical review of conformational B-cell epitope prediction methods

TL;DR

This study rigorously benchmarks nine conformational B-cell epitope prediction tools, revealing their generally poor performance and highlighting the need for improved methods and evaluation standards in the field.

Contribution

It provides a comprehensive benchmarking of existing tools on a large dataset and demonstrates their limitations, especially in the context of SARS-CoV-2 spike protein.

Findings

All methods show very low performance.

Consensus strategies are only marginally better than random.

Tools perform poorly on SARS-CoV-2 spike protein.

Abstract

Accurate in-silico prediction of conformational B-cell epitopes would lead to major improvements in disease diagnostics, drug design and vaccine development. A variety of computational methods, mainly based on machine learning approaches, have been developed in the last decades to tackle this challenging problem. Here, we rigorously benchmarked nine state-of-the-art conformational B-cell epitope prediction webservers, including generic and antibody-specific methods, on a dataset of over 250 antibody-antigen structures. The results of our assessment and statistical analyses show that all the methods achieve very low performances, and some do not perform better than randomly generated patches of surface residues. In addition, we also found that commonly used consensus strategies that combine the results from multiple webservers are at best only marginally better than random. Finally, we applied all the predictors to the SARS-CoV-2 spike protein as an independent case study, and showed that they perform poorly in general, which largely recapitulates our benchmarking conclusions. We hope that these results will lead to greater caution when using these tools until the biases and issues that limit current methods have been addressed, promote the use of state-of-the-art evaluation methodologies in future publications, and suggest new strategies to improve the performance of conformational B-cell epitope prediction methods.