LAFA: A Framework for Reproducible Longitudinal Assessment of Protein Function Annotation Models

TL;DR

LAFA is a persistent benchmarking platform that enables continuous, reproducible evaluation of protein function prediction models, tracking progress over time with evolving ground truth.

Contribution

It introduces LAFA, a system for ongoing assessment of protein function prediction methods, complementing periodic challenges like CAFA.

Findings

LAFA provides up-to-date performance comparisons of methods.

It supports reproducibility and continuous evaluation.

LAFA accelerates methodological development in protein function prediction.

Abstract

Motivation: Protein function prediction is a challenging task and an open problem in computational biology. The Critical Assessment of protein Function Annotation (CAFA) is a triennial, community-driven initiative that provides an independent, large-scale evaluation of computational methods for protein function prediction through time-delayed benchmarking experiments. CAFA has played a key role in highlighting high-performing methodologies and fostering detailed analysis and exchange of ideas. However, outside the periodic CAFA challenges, there is no platform for the continuous evaluation of newly developed methods and tracking performance as function annotations accumulate. Results: Here we introduce the Longitudinal Assessment of Protein Function Annotation Models server (LAFA) as a persistent benchmarking system for protein function prediction methods. LAFA provides a continuous evaluation of containerized function prediction methods, enabling up-to-date and robust comparative assessment of method performance under evolving ground truth. LAFA accelerates methodological iteration, supports reproducibility, and offers a more dynamic and fine-grained view of progress in protein function prediction. Code and Data Availability: LAFA is available at https://functionbench.net/. Detailed evaluation results can be found at https://github.com/anphan0828/CAFA_forever