Variational Search Distributions

TL;DR

VSD is a variational inference-based method for efficiently conditioning generative models on rare classes in discrete design spaces, applicable to biological sequence design and other combinatorial tasks.

Contribution

The paper introduces VSD, a novel active generation approach that leverages variational inference and scalable models for conditioned generative design.

Findings

VSD outperforms baseline methods on biological sequence design tasks.

The method demonstrates scalable and efficient learning of conditional distributions.

Asymptotic convergence rates are derived for the proposed variational approach.

Abstract

We develop VSD, a method for conditioning a generative model of discrete, combinatorial designs on a rare desired class by efficiently evaluating a black-box (e.g. experiment, simulation) in a batch sequential manner. We call this task active generation; we formalize active generation's requirements and desiderata, and formulate a solution via variational inference. VSD uses off-the-shelf gradient based optimization routines, can learn powerful generative models for desirable designs, and can take advantage of scalable predictive models. We derive asymptotic convergence rates for learning the true conditional generative distribution of designs with certain configurations of our method. After illustrating the generative model on images, we empirically demonstrate that VSD can outperform existing baseline methods on a set of real sequence-design problems in various protein and DNA/RNA engineering tasks.