D2Vformer: A Flexible Time Series Prediction Model Based on Time Position Embedding

TL;DR

D2Vformer is a novel time series prediction model that effectively captures complex temporal positional information using time position embeddings and attention mechanisms, outperforming existing methods on multiple datasets.

Contribution

The paper introduces D2Vformer, a flexible prediction model that directly handles variable-length sequences and improves time position embedding utilization with a new fusion block.

Findings

D2Vformer outperforms existing methods on six datasets.

Date2Vec embeddings surpass other time position embedding techniques.

The model reduces training resources compared to traditional approaches.

Abstract

Time position embeddings capture the positional information of time steps, often serving as auxiliary inputs to enhance the predictive capabilities of time series models. However, existing models exhibit limitations in capturing intricate time positional information and effectively utilizing these embeddings. To address these limitations, this paper proposes a novel model called D2Vformer. Unlike typical prediction methods that rely on RNNs or Transformers, this approach can directly handle scenarios where the predicted sequence is not adjacent to the input sequence or where its length dynamically changes. In comparison to conventional methods, D2Vformer undoubtedly saves a significant amount of training resources. In D2Vformer, the Date2Vec module uses the timestamp information and feature sequences to generate time position embeddings. Afterward, D2Vformer introduces a new fusion block that utilizes an attention mechanism to explore the similarity in time positions between the embeddings of the input sequence and the predicted sequence, thereby generating predictions based on this similarity. Through extensive experiments on six datasets, we demonstrate that Date2Vec outperforms other time position embedding methods, and D2Vformer surpasses state-of-the-art methods in both fixed-length and variable-length prediction tasks.