Importance Sampling via Score-based Generative Models

TL;DR

This paper introduces a training-free importance sampling method leveraging score-based generative models, enabling efficient sampling with multiple importance weights without additional training, applicable to diverse datasets and tasks.

Contribution

The paper presents a novel importance sampling framework that uses score-based generative models to perform sampling without training, accommodating multiple importance weights.

Findings

Effective across diverse datasets and tasks

Scalable and efficient importance sampling process

First method to handle multiple importance weights without retraining

Abstract

Importance sampling, which involves sampling from a probability density function (PDF) proportional to the product of an importance weight function and a base PDF, is a powerful technique with applications in variance reduction, biased or customized sampling, data augmentation, and beyond. Inspired by the growing availability of score-based generative models (SGMs), we propose an entirely training-free Importance sampling framework that relies solely on an SGM for the base PDF. Our key innovation is realizing the importance sampling process as a backward diffusion process, expressed in terms of the score function of the base PDF and the specified importance weight function--both readily available--eliminating the need for any additional training. We conduct a thorough analysis demonstrating the method's scalability and effectiveness across diverse datasets and tasks, including importance sampling for industrial and natural images with neural importance weight functions. The training-free aspect of our method is particularly compelling in real-world scenarios where a single base distribution underlies multiple biased sampling tasks, each requiring a different importance weight function. To the best of our knowledge our approach is the first importance sampling framework to achieve this.