Latent Space Bayesian Optimization with Latent Data Augmentation for Enhanced Exploration

TL;DR

This paper introduces a novel approach to enhance Latent Space Bayesian Optimization by addressing latent inconsistency through data augmentation and a new acquisition function, leading to improved exploration and sample efficiency in generative tasks.

Contribution

The paper proposes LCA-VAE and LCA-AF to improve LSBO by increasing latent consistency, resulting in more effective exploration and sample efficiency.

Findings

Enhanced exploration in de-novo image generation

Improved chemical design performance

Higher sample efficiency in experiments

Abstract

Latent Space Bayesian Optimization (LSBO) combines generative models, typically Variational Autoencoders (VAE), with Bayesian Optimization (BO) to generate de-novo objects of interest. However, LSBO faces challenges due to the mismatch between the objectives of BO and VAE, resulting in poor exploration capabilities. In this paper, we propose novel contributions to enhance LSBO efficiency and overcome this challenge. We first introduce the concept of latent consistency/inconsistency as a crucial problem in LSBO, arising from the VAE-BO mismatch. To address this, we propose the Latent Consistent Aware-Acquisition Function (LCA-AF) that leverages consistent points in LSBO. Additionally, we present LCA-VAE, a novel VAE method that creates a latent space with increased consistent points through data augmentation in latent space and penalization of latent inconsistencies. Combining LCA-VAE and LCA-AF, we develop LCA-LSBO. Our approach achieves high sample-efficiency and effective exploration, emphasizing the significance of addressing latent consistency through the novel incorporation of data augmentation in latent space within LCA-VAE in LSBO. We showcase the performance of our proposal via de-novo image generation and de-novo chemical design tasks.