RDPI: A Refine Diffusion Probability Generation Method for Spatiotemporal Data Imputation

TL;DR

RDPI introduces a two-stage diffusion-based framework that improves spatiotemporal data imputation accuracy and efficiency by combining deterministic initial estimates with residual refinement through a conditional diffusion model.

Contribution

The paper proposes a novel two-stage RDPI framework that enhances spatiotemporal data imputation by integrating deterministic methods with a residual diffusion process, addressing errors in autoregressive and simple diffusion models.

Findings

Achieves state-of-the-art imputation accuracy on multiple datasets.

Reduces sampling computational costs significantly.

Effectively captures spatiotemporal relationships in missing data.

Abstract

Spatiotemporal data imputation plays a crucial role in various fields such as traffic flow monitoring, air quality assessment, and climate prediction. However, spatiotemporal data collected by sensors often suffer from temporal incompleteness, and the sparse and uneven distribution of sensors leads to missing data in the spatial dimension. Among existing methods, autoregressive approaches are prone to error accumulation, while simple conditional diffusion models fail to adequately capture the spatiotemporal relationships between observed and missing data. To address these issues, we propose a novel two-stage Refined Diffusion Probability Impuation (RDPI) framework based on an initial network and a conditional diffusion model. In the initial stage, deterministic imputation methods are used to generate preliminary estimates of the missing data. In the refinement stage, residuals are treated as the diffusion target, and observed values are innovatively incorporated into the forward process. This results in a conditional diffusion model better suited for spatiotemporal data imputation, bridging the gap between the preliminary estimates and the true values. Experiments on multiple datasets demonstrate that RDPI not only achieves state-of-the-art imputation accuracy but also significantly reduces sampling computational costs.