Transfer learning on protein language models improves antimicrobial peptide classification

TL;DR

This paper shows that using large pre-trained protein models improves the classification of antimicrobial peptides, even with limited labeled data.

Contribution

The study demonstrates that transfer learning with protein language models achieves state-of-the-art AMP classification performance with minimal effort.

Findings

Model scale significantly improves AMP classification performance.

PLM embeddings with shallow classifiers achieve state-of-the-art results.

Fine-tuning PLM parameters further enhances classification accuracy.

Abstract

Antimicrobial peptides (AMPs) are essential components of the innate immune system in humans and other organisms, exhibiting potent activity against a broad spectrum of pathogens. Their potential therapeutic applications, particularly in combating antibiotic resistance, have rendered AMP classification a vital task in computational biology. However, the scarcity of labeled AMP sequences, coupled with the diversity and complexity of AMPs, poses significant challenges for the training of standalone AMP classifiers. Self-supervised learning has emerged as a powerful paradigm in addressing such challenges across various fields, leading to the development of Protein Language Models (PLMs). These models leverage vast amounts of unlabeled protein sequences to learn biologically relevant features, providing transferable protein sequence representations (embeddings), that can be fine-tuned for downstream tasks even with limited labeled data. This study evaluates the performance of several publicly-available PLMs in AMP classification utilizing transfer learning techniques and benchmarking them against state-of-the-art neural-based classifiers. Our key findings include: (a) Model scale is crucial, with classification performance consistently improving with increasing model size; (b) State-of-the-art results are achieved with minimal effort utilizing PLM embedding representations alongside shallow classifiers; and (c) Classification performance is further enhanced through efficient fine-tuning of PLMs’ parameters. Code showcasing our pipelines is available at https://github.com/EliasGeorg/PLM_AMP_Classification.

Figures