Learning Extrapolative Sequence Transformations from Markov Chains

TL;DR

This paper introduces a learned autoregressive model trained on Markov chain data to efficiently extrapolate sequence properties, outperforming traditional MCMC in scalability and sample efficiency across biological and text applications.

Contribution

It presents a novel method to learn an autoregressive model from Markov chain data, enabling effective extrapolation of sequence properties with fewer steps and greater efficiency.

Findings

Model outperforms MCMC in extrapolation tasks

Achieves higher sample efficiency and scalability

Validated on protein design and text tasks

Abstract

Most successful applications of deep learning involve similar training and test conditions. However, tasks such as biological sequence design involve searching for sequences that improve desirable properties beyond previously known values, which requires novel hypotheses that \emph{extrapolate} beyond training data. In these settings, extrapolation may be achieved by using random search methods such as Markov chain Monte Carlo (MCMC), which, given an initial state, sample local transformations to approximate a target density that rewards states with the desired properties. However, even with a well-designed proposal, MCMC may struggle to explore large structured state spaces efficiently. Rather than relying on stochastic search, it would be desirable to have a model that greedily optimizes the properties of interest, successfully extrapolating in as few steps as possible. We propose to learn such a model from the Markov chains resulting from MCMC search. Specifically, our approach uses selected states from Markov chains as a source of training data for an autoregressive model, which is then able to efficiently generate novel sequences that extrapolate along the sequence-level properties of interest. The proposed approach is validated on three problems: protein sequence design, text sentiment control, and text anonymization. We find that the autoregressive model can extrapolate as well or better than MCMC, but with the additional benefits of scalability and significantly higher sample efficiency.