Sequential Posterior Sampling with Diffusion Models

TL;DR

This paper introduces a novel transition model for diffusion models that accelerates posterior sampling in sequential imaging tasks, enabling real-time inference with minimal performance loss.

Contribution

It proposes a sequence-aware transition model using ViViT to reduce diffusion iterations, significantly speeding up inference in real-time applications.

Findings

Achieves 25x faster inference while maintaining performance.

Improves PSNR by up to 8% in motion-heavy sequences.

Demonstrates real-time posterior sampling in cardiac ultrasound imaging.

Abstract

Diffusion models have quickly risen in popularity for their ability to model complex distributions and perform effective posterior sampling. Unfortunately, the iterative nature of these generative models makes them computationally expensive and unsuitable for real-time sequential inverse problems such as ultrasound imaging. Considering the strong temporal structure across sequences of frames, we propose a novel approach that models the transition dynamics to improve the efficiency of sequential diffusion posterior sampling in conditional image synthesis. Through modeling sequence data using a video vision transformer (ViViT) transition model based on previous diffusion outputs, we can initialize the reverse diffusion trajectory at a lower noise scale, greatly reducing the number of iterations required for convergence. We demonstrate the effectiveness of our approach on a real-world dataset of high frame rate cardiac ultrasound images and show that it achieves the same performance as a full diffusion trajectory while accelerating inference 25$\times$, enabling real-time posterior sampling. Furthermore, we show that the addition of a transition model improves the PSNR up to 8\% in cases with severe motion. Our method opens up new possibilities for real-time applications of diffusion models in imaging and other domains requiring real-time inference.