SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space

TL;DR

This paper introduces SPD-DDPM, a novel generative diffusion model for symmetric positive definite matrices that effectively captures data distributions and makes accurate predictions, addressing scalability issues of previous methods.

Contribution

The paper proposes SPD-DDPM, a new diffusion-based generative model for SPD matrices that can learn both conditional and unconditional distributions efficiently.

Findings

Effective data distribution modeling demonstrated on toy and real taxi data.

Model accurately predicts $E(X|y)$ and generates samples matching data distribution.

Deeper SPD net improves conditional factor inclusion.

Abstract

Symmetric positive definite~(SPD) matrices have shown important value and applications in statistics and machine learning, such as FMRI analysis and traffic prediction. Previous works on SPD matrices mostly focus on discriminative models, where predictions are made directly on $E(X|y)$, where $y$ is a vector and $X$ is an SPD matrix. However, these methods are challenging to handle for large-scale data, as they need to access and process the whole data. In this paper, inspired by denoising diffusion probabilistic model~(DDPM), we propose a novel generative model, termed SPD-DDPM, by introducing Gaussian distribution in the SPD space to estimate $E(X|y)$. Moreover, our model is able to estimate $p(X)$ unconditionally and flexibly without giving $y$. On the one hand, the model conditionally learns $p(X|y)$ and utilizes the mean of samples to obtain $E(X|y)$ as a prediction. On the other hand, the model unconditionally learns the probability distribution of the data $p(X)$ and generates samples that conform to this distribution. Furthermore, we propose a new SPD net which is much deeper than the previous networks and allows for the inclusion of conditional factors. Experiment results on toy data and real taxi data demonstrate that our models effectively fit the data distribution both unconditionally and unconditionally and provide accurate predictions.