Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data

TL;DR

This paper introduces GAMA, an attribution method for generative sequence models trained only on positive data, enabling interpretability and biological insight extraction without negative data.

Contribution

The paper presents GAMA, a novel attribution technique for autoregressive generative models that works with positive-only data, filling a key gap in biological sequence analysis.

Findings

GAMA accurately recovers known biological features in synthetic datasets.

GAMA provides meaningful interpretability for antibody-antigen binding data.

The method enables validation of sequence design strategies without negative data.

Abstract

Generative machine learning models offer a powerful framework for therapeutic design by efficiently exploring large spaces of biological sequences enriched for desirable properties. Unlike supervised learning methods, which require both positive and negative labeled data, generative models such as LSTMs can be trained solely on positively labeled sequences, for example, high-affinity antibodies. This is particularly advantageous in biological settings where negative data are scarce, unreliable, or biologically ill-defined. However, the lack of attribution methods for generative models has hindered the ability to extract interpretable biological insights from such models. To address this gap, we developed Generative Attribution Metric Analysis (GAMA), an attribution method for autoregressive generative models based on Integrated Gradients. We assessed GAMA using synthetic datasets with known ground truths to characterize its statistical behavior and validate its ability to recover biologically relevant features. We further demonstrated the utility of GAMA by applying it to experimental antibody-antigen binding data. GAMA enables model interpretability and the validation of generative sequence design strategies without the need for negative training data.