TCRTransBench: A Comprehensive Benchmark for Bidirectional TCR-Peptide Sequence Generation

TL;DR

TCRTransBench introduces a standardized benchmark with curated datasets and evaluation metrics for bidirectional TCR-peptide sequence generation, facilitating progress in immunological modeling and therapeutic design.

Contribution

It provides the first comprehensive, curated dataset and evaluation framework for TCR-peptide sequence generation tasks, enabling systematic benchmarking of neural architectures.

Findings

Transformers outperform other models in capturing biological interactions.

Biologically informed evaluation metrics reveal key performance trade-offs.

The benchmark facilitates future research in immunological sequence modeling.

Abstract

T-cell receptor (TCR) interactions with antigenic peptides underpin adaptive immunity and are pivotal for personalized immunotherapy and vaccine development. Despite recent progress, computational modeling of TCR-peptide specificity remains challenging due to data scarcity, complex sequence dependencies, and the absence of standardized evaluation frameworks. To systematically address these issues, we introduce TCRTransBench, a comprehensive benchmark for bidirectional TCR-peptide sequence generation tasks. Specifically, we define two sequence-to-sequence (seq2seq) tasks: generating antigenic peptides from TCR sequences (TCR2PEP) and generating TCR sequences from antigenic peptides (PEP2TCR). Our framework provides a rigorously curated, MHC-free dataset comprising tens of thousands of validated TCR-peptide pairs, along with diverse evaluation metrics that integrate computational efficiency, sequence accuracy, and biological plausibility. Extensive benchmarking across representative neural architectures, including recurrent, convolutional, and transformer-based models, reveals key trade-offs among performance metrics, highlighting the effectiveness of transformers in capturing intricate biological interactions and the necessity of biologically informed evaluation criteria. TCRTransBench establishes standardized tasks, datasets, and evaluation protocols, laying a robust foundation for future computational advances in immunological sequence modeling and therapeutic protein design.