Deep Temporal Deaggregation: Large-Scale Spatio-Temporal Generative Models

TL;DR

This paper introduces TDDPM, a transformer-based diffusion model for large-scale, long-sequence time-series data, particularly mobility data, addressing scalability, accuracy, and adaptability challenges in generative modeling.

Contribution

It presents a novel scalable transformer-based diffusion model for long and large-scale time-series, with a comprehensive benchmark and capabilities for conditional and out-of-distribution generation.

Findings

Outperforms state-of-the-art models in scalability and accuracy.

Effectively generates mobility data for unseen environments.

Demonstrates robust out-of-distribution generalization.

Abstract

Many of today's data is time-series data originating from various sources, such as sensors, transaction systems, or production systems. Major challenges with such data include privacy and business sensitivity. Generative time-series models have the potential to overcome these problems, allowing representative synthetic data, such as people's movement in cities, to be shared openly and be used to the benefit of society at large. However, contemporary approaches are limited to prohibitively short sequences and small scales. Aside from major memory limitations, the models generate less accurate and less representative samples the longer the sequences are. This issue is further exacerbated by the lack of a comprehensive and accessible benchmark. Furthermore, a common need in practical applications is what-if analysis and dynamic adaptation to data distribution changes, for usage in decision making and to manage a changing world: What if this road is temporarily blocked or another road is added? The focus of this paper is on mobility data, such as people's movement in cities, requiring all these issues to be addressed. To this end, we propose a transformer-based diffusion model, TDDPM, for time-series which outperforms and scales substantially better than state-of-the-art. This is evaluated in a new comprehensive benchmark across several sequence lengths, standard datasets, and evaluation measures. We also demonstrate how the model can be conditioned on a prior over spatial occupancy frequency information, allowing the model to generate mobility data for previously unseen environments and for hypothetical scenarios where the underlying road network and its usage changes. This is evaluated by training on mobility data from part of a city. Then, using only aggregate spatial information as prior, we demonstrate out-of-distribution generalization to the unobserved remainder of the city.