A Study of Posterior Stability for Time-Series Latent Diffusion
Yangming Li, Yixin Cheng, Mihaela van der Schaar
TL;DR
This paper investigates posterior collapse in time-series latent diffusion models, introduces a dependency measure to quantify sensitivity, and proposes a new stable framework that outperforms existing methods in real datasets.
Contribution
It identifies the issue of posterior collapse in time-series latent diffusion, introduces a dependency measure to analyze it, and develops a new framework that ensures posterior stability and improved performance.
Findings
Posterior collapse reduces latent diffusion to a less expressive VAE.
Dependency measure effectively quantifies sensitivity of recurrent decoders.
New framework achieves stable posterior and outperforms baselines in real datasets.
Abstract
Latent diffusion has demonstrated promising results in image generation and permits efficient sampling. However, this framework might suffer from the problem of posterior collapse when applied to time series. In this paper, we first show that posterior collapse will reduce latent diffusion to a variational autoencoder (VAE), making it less expressive. This highlights the importance of addressing this issue. We then introduce a principled method: dependency measure, that quantifies the sensitivity of a recurrent decoder to input variables. Using this tool, we confirm that posterior collapse significantly affects time-series latent diffusion on real datasets, and a phenomenon termed dependency illusion is also discovered in the case of shuffled time series. Finally, building on our theoretical and empirical studies, we introduce a new framework that extends latent diffusion and has a…
Peer Reviews
Decision·Submitted to ICLR 2025
1. The introduction of dependency measures to diagnose and address posterior collapse is both novel and insightful, providing a fresh perspective on an important issue within latent diffusion models. 2. The paper offers a solid theoretical foundation for the analysis of posterior collapse, and the proposed framework is well-motivated by both theoretical insights and empirical observations. 3. The proposed framework demonstrates significant improvements in the performance of time-series generatio
1. While the paper presents strong results for time-series data, it lacks a detailed discussion on the generalizability of the approach to other data modalities, such as images or text. Including a brief exploration or discussion of potential extensions could further enhance the contribution. 2. The experimental details, including specific configurations for baselines and the selection of hyperparameters, are not fully elaborated in the main text. Providing more comprehensive explanations in the
- The paper addresses a previously underexplored issue in time series diffusion models—posterior collapse—which has primarily been studied in variational autoencoders (VAEs) but not in the context of diffusion models for time series. - The dependency measure provides an insightful tool for quantifying the decoder’s reliance on the latent variable. This measure enables detection of both posterior collapse and dependency illusion, offering valuable diagnostic capabilities for latent-variable model
- The empirical evaluation lacks comparisons with stable time series models that naturally avoid posterior collapse, such as ARIMA, RNNs, LSTMs, transformers, and temporal convolutional networks. Including these baselines would provide context on whether the proposed framework offers advantages beyond mitigating posterior collapse. The author also did not compare with recent baselines for time series, which are diffusion-based. Please check papers published in NeurIPS/ICLR/ICML in the past two y
--The paper tries to focus on the specific issues of time-dependency collapse in the case of time series data and diffusion models. --The shuffling experiments help illustrate how a latent variable is not being used strongly throughout all time steps
--The problem is not sufficiently well motivated. In particular, the two types of mode collapse which in time series (time-dependent and time-independent) are not discussed. The reduction to a VAE is only about the elimination of the time-dependent influence. The impact of this simplification is not sufficiently discussed. --Moreover, the less expressivity is not shown explicitly to be a bad thing in the context of time series in general. There are potentially time series which are driven b
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTime Series Analysis and Forecasting
MethodsDiffusion