DBT-DMAE: An Effective Multivariate Time Series Pre-Train Model under Missing Data

TL;DR

This paper introduces DBT-DMAE, a novel multivariate time series pre-training model that effectively handles missing data, improving downstream task performance across various datasets and tasks.

Contribution

The paper proposes a new pre-training model with a missing representation module and a dynamic-bidirectional TCN, addressing missing data issues in multivariate time series.

Findings

DBT-DMAE outperforms state-of-the-art methods on six real-world datasets.

The model effectively captures temporal features with dynamic kernels.

Ablation studies confirm the importance of each substructure.

Abstract

Multivariate time series(MTS) is a universal data type related to many practical applications. However, MTS suffers from missing data problems, which leads to degradation or even collapse of the downstream tasks, such as prediction and classification. The concurrent missing data handling procedures could inevitably arouse the biased estimation and redundancy-training problem when encountering multiple downstream tasks. This paper presents a universally applicable MTS pre-train model, DBT-DMAE, to conquer the abovementioned obstacle. First, a missing representation module is designed by introducing dynamic positional embedding and random masking processing to characterize the missing symptom. Second, we proposed an auto-encoder structure to obtain the generalized MTS encoded representation utilizing an ameliorated TCN structure called dynamic-bidirectional-TCN as the basic unit, which integrates the dynamic kernel and time-fliping trick to draw temporal features effectively. Finally, the overall feed-in and loss strategy is established to ensure the adequate training of the whole model. Comparative experiment results manifest that the DBT-DMAE outperforms the other state-of-the-art methods in six real-world datasets and two different downstream tasks. Moreover, ablation and interpretability experiments are delivered to verify the validity of DBT-DMAE's substructures.