Diffusion Model for Data-Driven Black-Box Optimization

TL;DR

This paper introduces a diffusion model-based approach for black-box optimization that leverages unlabeled and limited labeled data to generate near-optimal structured designs, with theoretical guarantees and empirical validation.

Contribution

It proposes a reward-directed conditional diffusion model for black-box optimization, establishing sub-optimality bounds and demonstrating efficiency in high-dimensional and low-dimensional latent spaces.

Findings

The model achieves near-optimal solutions with sub-optimality gaps close to theoretical bounds.

It effectively handles noisy reward measurements and human preference data.

Empirical results validate the model's performance in decision-making and content-creation tasks.

Abstract

Generative AI has redefined artificial intelligence, enabling the creation of innovative content and customized solutions that drive business practices into a new era of efficiency and creativity. In this paper, we focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization over complex structured variables. Consider the practical scenario where one wants to optimize some structured design in a high-dimensional space, based on massive unlabeled data (representing design variables) and a small labeled dataset. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. The goal is to generate new designs that are near-optimal and preserve the designed latent structures. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models for modeling complex distributions. In particular, we propose a reward-directed conditional diffusion model, to be trained on the mixed data, for sampling a near-optimal solution conditioned on high predicted rewards. Theoretically, we establish sub-optimality error bounds for the generated designs. The sub-optimality gap nearly matches the optimal guarantee in off-policy bandits, demonstrating the efficiency of reward-directed diffusion models for black-box optimization. Moreover, when the data admits a low-dimensional latent subspace structure, our model efficiently generates high-fidelity designs that closely respect the latent structure. We provide empirical experiments validating our model in decision-making and content-creation tasks.