Time-Transformer: Integrating Local and Global Features for Better Time Series Generation (Extended Version)

TL;DR

Time-Transformer AAE is a novel generative model that effectively captures both local and global features in time series data, improving data generation quality especially for complex datasets.

Contribution

The paper introduces the Time-Transformer architecture within an adversarial autoencoder, combining convolutional and transformer-based modules with cross attention for enhanced time series generation.

Findings

Outperforms state-of-the-art models on 5 out of 6 datasets

Effective in generating data with both local and global properties

Supports data augmentation for small and imbalanced datasets

Abstract

Generating time series data is a promising approach to address data deficiency problems. However, it is also challenging due to the complex temporal properties of time series data, including local correlations as well as global dependencies. Most existing generative models have failed to effectively learn both the local and global properties of time series data. To address this open problem, we propose a novel time series generative model named 'Time-Transformer AAE', which consists of an adversarial autoencoder (AAE) and a newly designed architecture named 'Time-Transformer' within the decoder. The Time-Transformer first simultaneously learns local and global features in a layer-wise parallel design, combining the abilities of Temporal Convolutional Networks and Transformer in extracting local features and global dependencies respectively. Second, a bidirectional cross attention is proposed to provide complementary guidance across the two branches and achieve proper fusion between local and global features. Experimental results demonstrate that our model can outperform existing state-of-the-art models in 5 out of 6 datasets, specifically on those with data containing both global and local properties. Furthermore, we highlight our model's advantage on handling this kind of data via an artificial dataset. Finally, we show our model's ability to address a real-world problem: data augmentation to support learning with small datasets and imbalanced datasets.