A Multi-Task Learning Approach to Linear Multivariate Forecasting

TL;DR

This paper introduces a multi-task learning framework for linear multivariate forecasting that considers inter-variate relationships, uses correlation-based clustering for task grouping, and balances gradients to improve forecasting accuracy.

Contribution

It proposes a novel multi-task learning approach for multivariate forecasting that incorporates task grouping and gradient balancing, outperforming or matching existing methods.

Findings

Achieves on-par or better results than strong baselines.

Uses correlation-based clustering for task grouping.

Demonstrates effectiveness on challenging benchmarks.

Abstract

Accurate forecasting of multivariate time series data is important in many engineering and scientific applications. Recent state-of-the-art works ignore the inter-relations between variates, using their model on each variate independently. This raises several research questions related to proper modeling of multivariate data. In this work, we propose to view multivariate forecasting as a multi-task learning problem, facilitating the analysis of forecasting by considering the angle between task gradients and their balance. To do so, we analyze linear models to characterize the behavior of tasks. Our analysis suggests that tasks can be defined by grouping similar variates together, which we achieve via a simple clustering that depends on correlation-based similarities. Moreover, to balance tasks, we scale gradients with respect to their prediction error. Then, each task is solved with a linear model within our MTLinear framework. We evaluate our approach on challenging benchmarks in comparison to strong baselines, and we show it obtains on-par or better results on multivariate forecasting problems. The implementation is available at: https://github.com/azencot-group/MTLinear