One protein is all you need
Anton Bushuiev, Roman Bushuiev, Olga Pimenova, Nikola Zadorozhny, Raman Samusevich, Elisabet Manaskova, Rachel Seongeun Kim, Hannes St\"ark, Jiri Sedlar, Martin Steinegger, Tom\'a\v{s} Pluskal, Josef Sivic
TL;DR
This paper introduces ProteinTTT, a novel method for customizing protein language models to individual target proteins on-the-fly, significantly improving prediction accuracy for specific proteins without additional data.
Contribution
The paper presents ProteinTTT, a self-supervised test-time training approach that enhances model performance on specific proteins, addressing limitations of generalization in protein modeling.
Findings
ProteinTTT improves structure prediction for challenging targets.
Achieves new state-of-the-art in protein fitness prediction.
Enhances function prediction and antibody-antigen modeling.
Abstract
Generalization beyond training data remains a central challenge in machine learning for biology. A common way to enhance generalization is self-supervised pre-training on large datasets. However, aiming to perform well on all possible proteins can limit a model's capacity to excel on any specific one, whereas experimentalists typically need accurate predictions for individual proteins they study, often not covered in training data. To address this limitation, we propose a method that enables self-supervised customization of protein language models to one target protein at a time, on the fly, and without assuming any additional data. We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets. ProteinTTT improves structure prediction for challenging targets, achieves new state-of-the-art results…
Peer Reviews
Decision·ICLR 2026 Poster
1. The method is simple and general. It does not require additional data and can be easily applied to different pretrained PLMs. And the appraoch leverages lightweight finetuning (LoRA and limited steps), making it resource-efficient. 2. The method consistently improve the performance on different tasks and different models. 3. The authors use case studies to demonstrate real-world applications. For biologists and chemists who are interested in specific proteins, this could be useful.
1. The authors claimed that the method can be easily extended to models with autoregressive masking, while they did not do that. I suggest the authors might try to show the results on ProGen. 2. The improvement on larger models are marginal. While ProteinTTT consistently improve results, the improvement on larger model is marginal and even negligible. For example, in table 2 when it comes to 650M model. The improvement is nearly negligible. I wonder what will happen in larger model like 1B or 3B
1. The manuscript is well written and polished; clear, concise, and detailed enough in both method and evaluation to be easy to follow. 2. The experimental design is comprehensive, covering three relevant downstream tasks with appropriate baselines. The inclusion of case studies and in-depth analyses further strengthens the empirical validation. 3. The results are consistent and convincing, showing that ProteinTTT can effectively enhance the performance of existing protein language models. 4. In
1. The proposed method seems inherently limited in scope, it applies only to protein language models and only to sequence-based tasks. This restricts its applicability to broader classes of protein modeling problems. 2. While the reported improvements are consistent across benchmarks, many gains are modest and appear bounded by the baseline model’s performance. The authors briefly acknowledge this (e.g., in G1), but a clearer discussion in the main text along with analysis of failure cases would
The manuscript is exceptionally clearly written, and the method is intuitive and easy to follow. The authors include a diverse set of experiments run on several different models, and I appreciate the inclusion of confidence intervals throughout. Methods like ProteinTTT obviously have long pedigrees in the language modeling and computer vision literature, but this is the first paper I'm aware of that applies this sort of idea to protein modeling.
The most glaring weakness in my eyes is that the effect of adaptation seems to be relatively limited. On several benchmarks, improvements are measured as fractions of percentage points. On several others (especially in section 5), the improvement is more difficult to gauge; at various points, the authors make claims about the fractions of proteins that are improved by the method, but details about the magnitude of the improvement in these cases seem to be pretty sparse. Figure 5b hints that some
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Proteomics Techniques and Applications