Approximately Equivariant Recurrent Generative Models for Quasi-Periodic Time Series with a Progressive Training Scheme

TL;DR

This paper introduces AEQ-RVAE-ST, a recurrent variational autoencoder with a progressive training scheme that stabilizes learning of long, quasi-periodic time series, achieving state-of-the-art results.

Contribution

The paper proposes a novel training scheme and model architecture that improve stability and performance in generative modeling of long, quasi-periodic time series.

Findings

AEQ-RVAE-ST outperforms existing models on benchmark datasets.

The progressive training scheme stabilizes optimization for long sequences.

The model effectively captures quasi-periodic and stationary time series structures.

Abstract

We present a simple yet effective generative model for time series, based on a Recurrent Variational Autoencoder that we refer to as AEQ-RVAE-ST. Recurrent layers often struggle with unstable optimization and poor convergence when modeling long sequences. To address these limitations, we introduce a training scheme that subsequently increases the sequence length, stabilizing optimization and enabling consistent learning over extended horizons. By composing known components into a recurrent, approximately time-shift-equivariant topology, our model introduces an inductive bias that aligns with the structure of quasi-periodic and nearly stationary time series. Across several benchmark datasets, AEQ-RVAE-ST matches or surpasses state-of-the-art generative models, particularly on quasi-periodic data, while remaining competitive on more irregular signals. Performance is evaluated through ELBO, Fr\'echet Distance, discriminative metrics, and visualizations of the learned latent embeddings.