Pairing interacting protein sequences using masked language modeling

TL;DR

This paper introduces DiffPALM, a novel method leveraging protein language models to accurately pair interacting protein sequences, significantly improving structure prediction of protein complexes using deep learning.

Contribution

The paper presents DiffPALM, a differentiable approach that exploits MSA Transformer to pair protein sequences, outperforming existing methods on challenging benchmarks without requiring fine-tuning.

Findings

DiffPALM outperforms existing coevolution-based pairing methods.

It improves structure prediction accuracy of protein complexes with AlphaFold-Multimer.

The method works well even with shallow multiple sequence alignments.

Abstract

Predicting which proteins interact together from amino-acid sequences is an important task. We develop a method to pair interacting protein sequences which leverages the power of protein language models trained on multiple sequence alignments, such as MSA Transformer and the EvoFormer module of AlphaFold. We formulate the problem of pairing interacting partners among the paralogs of two protein families in a differentiable way. We introduce a method called DiffPALM that solves it by exploiting the ability of MSA Transformer to fill in masked amino acids in multiple sequence alignments using the surrounding context. MSA Transformer encodes coevolution between functionally or structurally coupled amino acids. We show that it captures inter-chain coevolution, while it was trained on single-chain data, which means that it can be used out-of-distribution. Relying on MSA Transformer without fine-tuning, DiffPALM outperforms existing coevolution-based pairing methods on difficult benchmarks of shallow multiple sequence alignments extracted from ubiquitous prokaryotic protein datasets. It also outperforms an alternative method based on a state-of-the-art protein language model trained on single sequences. Paired alignments of interacting protein sequences are a crucial ingredient of supervised deep learning methods to predict the three-dimensional structure of protein complexes. DiffPALM substantially improves the structure prediction of some eukaryotic protein complexes by AlphaFold-Multimer, without significantly deteriorating any of those we tested. It also achieves competitive performance with using orthology-based pairing.