PETRA: Pretrained Evolutionary Transformer for SARS-CoV-2 Mutation Prediction
Xu Zou
TL;DR
PETRA introduces a novel transformer model based on evolutionary trajectories from phylogenetic trees to predict SARS-CoV-2 mutations, effectively handling noisy data and capturing viral evolution hierarchies.
Contribution
The paper presents PETRA, a transformer leveraging phylogenetic information for mutation prediction, improving over existing models in noisy viral sequence data.
Findings
PETRA achieves higher weighted recall@1 for nucleotide and spike mutations.
It effectively mitigates sequencing noise and captures hierarchical viral evolution.
Demonstrates real-time mutation prediction for major SARS-CoV-2 clades.
Abstract
Since its emergence, SARS-CoV-2 has demonstrated a rapid and unpredictable evolutionary trajectory, characterized by the continual emergence of immune-evasive variants. This poses persistent challenges to public health and vaccine development. While large-scale generative pre-trained transformers (GPTs) have revolutionized the modeling of sequential data, their direct applications to noisy viral genomic sequences are limited. In this paper, we introduce PETRA(Pretrained Evolutionary TRAnsformer), a novel transformer approach based on evolutionary trajectories derived from phylogenetic trees rather than raw RNA sequences. This method effectively mitigates sequencing noise and captures the hierarchical structure of viral evolution. With a weighted training framework to address substantial geographical and temporal imbalances in global sequence data, PETRA excels in predicting future…
Peer Reviews
Decision·Submitted to ICLR 2026
A key strength of PETRA is that it addresses a major bottleneck in immunogen design: experimental characterization of mutational effects is slow, expensive, and inherently incomplete. The model’s zero-shot capability suggests it captures generalizable evolutionary principles rather than memorizing training examples. The authors emphasize that mutational fitness depends jointly on structural context and immune-driven selection, an important insight consistent with known antigenic drift dynamics.
One limitation is that the manuscript appears to focus primarily on sequence-based learning; explicit integration of three-dimensional structural context, epistatic coupling, or antibody–antigen interface geometry is not clearly articulated. Viral evolution is strongly epistatic, yet the evaluation setup seems to emphasize single-mutation effects, leaving open how well PETRA handles combinatorial variants observed in real variants of concern. The experimental validation section would benefit fro
- The time- and geography- based weighting is interesting and sounds more broadly applicable. - The way in which the sequence is encoded is interesting -- one-hot encoding each site x mutation pair and concaternating them. - Careful temporal train/test splits.
- The paper is poorly written in terms of grammar and phrasing. The abstract is split into three paragraphs. Grammatical errors are broadly present. Citation is non-standard, with citations after the period. An oddly harsh and dismissive phrase is used when referring to existing work: "There also exist researches attempting to build up transformer-based models directly for SARS-CoV-2. (Shou et al., 2023; Feng et al., 2024) Nevertheless, these attempts focus on specially framed datasets of seque
The paper is well written.
1. The model learns to extrapolate the pre-existing UShER tree, not viral evolution itself, making it useless for novel variants like Omicron where no such tree exists. It is pattern-matching on a graph, not learning biology. 2. The evaluation is critically flawed by the omission of direct comparisons to the actual state-of-the-art viral forecasting models discussed in recent scientific literature (e.g., scientific works mentioned in Nature News: https://www.nature.com/articles/d41586-024-04195-
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSARS-CoV-2 and COVID-19 Research · Genomics and Rare Diseases · vaccines and immunoinformatics approaches