# PLM-eXplain: divide and conquer the protein embedding space

## TL;DR

PLM-eXplain improves the interpretability of protein language models by splitting their embeddings into interpretable and predictive components, without losing accuracy.

## Contribution

PLM-eXplain introduces an explainable adapter layer that factors PLM embeddings into interpretable biochemical features and predictive residuals.

## Key findings

- PLM-X maintains high predictive performance while incorporating interpretable biochemical features like secondary structure and hydropathy.
- The method was successfully applied to three classification tasks: extracellular vesicle association, transmembrane helix prediction, and aggregation propensity prediction.
- PLM-X offers a generalizable solution for enhancing the interpretability of protein language models across various applications.

## Abstract

Protein language models (PLMs) have revolutionized computational biology through their ability to generate powerful sequence representations for diverse prediction tasks. However, their black-box nature limits biological interpretation and translation to actionable insights. Bridging this gap requires approaches that maintain predictive performance while providing interpretable explanations of model behaviour.

We present PLM-eXplain (PLM-X), an explainable adapter layer that bridges this gap by factoring PLM embeddings into two complementary components: an interpretable subspace based on established biochemical features, and a residual subspace that retains predictive, non-interpretable information. Using embeddings from ESM2 and ProtBert, PLM-X incorporates well-established properties, including secondary structure and hydropathy, while maintaining high predictive performance. We demonstrate the effectiveness of our approach across three biologically relevant classification tasks: extracellular vesicle association, transmembrane helix prediction, and aggregation propensity prediction. PLM-X enables biological interpretation of model decisions without sacrificing accuracy, offering a generalizable solution for enhancing PLM interpretability across various downstream applications.

Source code and models are available at https://github.com/AIT4LIFE-UU/PLM-eXplain/.