Exploring the design space of diffusion and flow models for data fusion

TL;DR

This paper investigates various diffusion and flow models for satellite data fusion, emphasizing the effectiveness of UNet-based diffusion models and noise scheduling strategies to improve image quality and computational efficiency.

Contribution

It systematically explores the design space of diffusion and flow models for data fusion, providing practical guidance and insights for remote sensing applications.

Findings

UNet-based diffusion models excel at preserving spatial details

Discrete noise schedulers yield higher-quality reconstructions

Quantization techniques improve memory and computational efficiency

Abstract

Data fusion is an essential task in various domains, enabling the integration of multi-source information to enhance data quality and insights. One key application is in satellite remote sensing, where fusing multi-sensor observations can improve spatial and temporal resolution. In this study, we explore the design space of diffusion and flow models for data fusion, focusing on the integration of Defense Meteorological Satellite Program's Operational Linescan System (DMSP-OLS) and Visible Infrared Imaging Radiometer Suite (VIIRS) nighttime lights data. Our approach leverages a diverse set of 2D image-to-image generative models, including UNET, diffusion, and flow modeling architectures. We evaluate the effectiveness of these architectures in satellite remote sensing data fusion, identifying diffusion models based on UNet as particularly adept at preserving fine-grained spatial details and generating high-fidelity fused images. We also provide guidance on the selection of noise schedulers in diffusion-based models, highlighting the trade-offs between iterative solvers for faster inference and discrete schedulers for higher-quality reconstructions. Additionally, we explore quantization techniques to optimize memory efficiency and computational cost without compromising performance. Our findings offer practical insights into selecting the most effective diffusion and flow model architectures for data fusion tasks, particularly in remote sensing applications, and provide recommendations for leveraging noise scheduling strategies to enhance fusion quality.