Proteina: Scaling Flow-based Protein Structure Generative Models

TL;DR

Proteina introduces a scalable flow-based model for protein backbone generation, leveraging hierarchical conditioning and advanced training techniques to produce diverse, long, and designable proteins with state-of-the-art performance.

Contribution

The paper presents Proteina, a large-scale flow-based protein generator with hierarchical conditioning, novel training strategies, and new metrics for evaluating protein structure generation.

Findings

Achieves state-of-the-art performance in de novo protein backbone design.

Generates diverse proteins up to 800 residues long.

Provides high-level control over secondary structures and fold-specific features.

Abstract

Recently, diffusion- and flow-based generative models of protein structures have emerged as a powerful tool for de novo protein design. Here, we develop Proteina, a new large-scale flow-based protein backbone generator that utilizes hierarchical fold class labels for conditioning and relies on a tailored scalable transformer architecture with up to 5x as many parameters as previous models. To meaningfully quantify performance, we introduce a new set of metrics that directly measure the distributional similarity of generated proteins with reference sets, complementing existing metrics. We further explore scaling training data to millions of synthetic protein structures and explore improved training and sampling recipes adapted to protein backbone generation. This includes fine-tuning strategies like LoRA for protein backbones, new guidance methods like classifier-free guidance and autoguidance for protein backbones, and new adjusted training objectives. Proteina achieves state-of-the-art performance on de novo protein backbone design and produces diverse and designable proteins at unprecedented length, up to 800 residues. The hierarchical conditioning offers novel control, enabling high-level secondary-structure guidance as well as low-level fold-specific generation.