Time Series Diffusion in the Frequency Domain

TL;DR

This paper investigates the use of Fourier analysis in time series diffusion models, introducing frequency domain diffusion, which better captures data distributions in real-world datasets like healthcare and finance.

Contribution

The paper develops a novel frequency domain diffusion framework for time series, adapting score matching, and demonstrates its advantages over traditional time domain models.

Findings

Frequency diffusion models outperform time diffusion models on real-world datasets.

Time series are more localized in the frequency domain, facilitating easier modeling.

Mirrored Brownian motions are key to dual diffusion processes in the frequency domain.

Abstract

Fourier analysis has been an instrumental tool in the development of signal processing. This leads us to wonder whether this framework could similarly benefit generative modelling. In this paper, we explore this question through the scope of time series diffusion models. More specifically, we analyze whether representing time series in the frequency domain is a useful inductive bias for score-based diffusion models. By starting from the canonical SDE formulation of diffusion in the time domain, we show that a dual diffusion process occurs in the frequency domain with an important nuance: Brownian motions are replaced by what we call mirrored Brownian motions, characterized by mirror symmetries among their components. Building on this insight, we show how to adapt the denoising score matching approach to implement diffusion models in the frequency domain. This results in frequency diffusion models, which we compare to canonical time diffusion models. Our empirical evaluation on real-world datasets, covering various domains like healthcare and finance, shows that frequency diffusion models better capture the training distribution than time diffusion models. We explain this observation by showing that time series from these datasets tend to be more localized in the frequency domain than in the time domain, which makes them easier to model in the former case. All our observations point towards impactful synergies between Fourier analysis and diffusion models.