Reconstructing Spatiotemporal Data with C-VAEs

TL;DR

This paper investigates the use of Conditional Variational Autoencoders (C-VAEs) to generate smooth, realistic, and temporally consistent representations of moving regions in spatiotemporal data, outperforming traditional interpolation methods.

Contribution

It introduces a C-VAE-based approach for spatiotemporal region interpolation, demonstrating improved temporal consistency over existing methods.

Findings

C-VAE achieves competitive geometric similarity metrics.

C-VAE exhibits superior temporal consistency.

Outperforms traditional interpolation algorithms.

Abstract

The continuous representation of spatiotemporal data commonly relies on using abstract data types, such as \textit{moving regions}, to represent entities whose shape and position continuously change over time. Creating this representation from discrete snapshots of real-world entities requires using interpolation methods to compute in-between data representations and estimate the position and shape of the object of interest at arbitrary temporal points. Existing region interpolation methods often fail to generate smooth and realistic representations of a region's evolution. However, recent advancements in deep learning techniques have revealed the potential of deep models trained on discrete observations to capture spatiotemporal dependencies through implicit feature learning. In this work, we explore the capabilities of Conditional Variational Autoencoder (C-VAE) models to generate smooth and realistic representations of the spatiotemporal evolution of moving regions. We evaluate our proposed approach on a sparsely annotated dataset on the burnt area of a forest fire. We apply compression operations to sample from the dataset and use the C-VAE model and other commonly used interpolation algorithms to generate in-between region representations. To evaluate the performance of the methods, we compare their interpolation results with manually annotated data and regions generated by a U-Net model. We also assess the quality of generated data considering temporal consistency metrics. The proposed C-VAE-based approach demonstrates competitive results in geometric similarity metrics. It also exhibits superior temporal consistency, suggesting that C-VAE models may be a viable alternative to modelling the spatiotemporal evolution of 2D moving regions.