Partial-Multivariate Model for Forecasting

TL;DR

This paper introduces PMformer, a Transformer-based partial-multivariate model for forecasting that balances between univariate and complete-multivariate approaches, improving accuracy, efficiency, and robustness.

Contribution

The paper proposes a novel partial-multivariate model, PMformer, with a training algorithm and inference technique, demonstrating its superiority over existing models in forecasting tasks.

Findings

PMformer outperforms univariate and complete-multivariate models in accuracy.

The model is more efficient and robust, especially with missing features.

Empirical results validate the theoretical advantages of the approach.

Abstract

When solving forecasting problems including multiple time-series features, existing approaches often fall into two extreme categories, depending on whether to utilize inter-feature information: univariate and complete-multivariate models. Unlike univariate cases which ignore the information, complete-multivariate models compute relationships among a complete set of features. However, despite the potential advantage of leveraging the additional information, complete-multivariate models sometimes underperform univariate ones. Therefore, our research aims to explore a middle ground between these two by introducing what we term Partial-Multivariate models where a neural network captures only partial relationships, that is, dependencies within subsets of all features. To this end, we propose PMformer, a Transformer-based partial-multivariate model, with its training algorithm. We demonstrate that PMformer outperforms various univariate and complete-multivariate models, providing a theoretical rationale and empirical analysis for its superiority. Additionally, by proposing an inference technique for PMformer, the forecasting accuracy is further enhanced. Finally, we highlight other advantages of PMformer: efficiency and robustness under missing features.