Prompt-Guided Injection of Conformation to Pre-trained Protein Model

TL;DR

This paper introduces a method to enhance pre-trained protein models by injecting interpretable prompts that incorporate conformational knowledge, improving their performance on structure-related tasks without sacrificing sequence-based task accuracy.

Contribution

The paper proposes a novel prompt-based approach to inject conformational knowledge into pre-trained protein models, enabling multi-task learning for sequence and structure-aware representations.

Findings

Interaction-conformation prompts improve structure-related task performance.

Sequence prompts do not affect sequence task performance.

Learned prompts can be combined for complex tasks.

Abstract

Pre-trained protein models (PTPMs) represent a protein with one fixed embedding and thus are not capable for diverse tasks. For example, protein structures can shift, namely protein folding, between several conformations in various biological processes. To enable PTPMs to produce task-aware representations, we propose to learn interpretable, pluggable and extensible protein prompts as a way of injecting task-related knowledge into PTPMs. In this regard, prior PTPM optimization with the masked language modeling task can be interpreted as learning a sequence prompt (Seq prompt) that enables PTPMs to capture the sequential dependency between amino acids. To incorporate conformational knowledge to PTPMs, we propose an interaction-conformation prompt (IC prompt) that is learned through back-propagation with the protein-protein interaction task. As an instantiation, we present a conformation-aware pre-trained protein model that learns both sequence and interaction-conformation prompts in a multi-task setting. We conduct comprehensive experiments on nine protein datasets. Results confirm our expectation that using the sequence prompt does not hurt PTPMs' performance on sequence-related tasks while incorporating the interaction-conformation prompt significantly improves PTPMs' performance on tasks where conformational knowledge counts. We also show the learned prompts can be combined and extended to deal with new complex tasks.