SUM: Suboptimal Unitary Multi-task Learning Framework for Spatiotemporal Data Prediction

TL;DR

This paper introduces SUM, a two-step suboptimal unitary method that enhances multi-task learning for spatiotemporal data prediction, especially with few tasks, by combining meta-learning with gradient descent.

Contribution

The paper proposes a novel SUM framework that integrates meta-learning into multi-task models, improving generalization and enabling coKriging, applicable to non-linear models with small task numbers.

Findings

SUM outperforms low-rank tensor methods with fewer tasks.

The framework improves prediction accuracy on public datasets.

It enables traditional models to perform coKriging effectively.

Abstract

The typical multi-task learning methods for spatio-temporal data prediction involve low-rank tensor computation. However, such a method have relatively weak performance when the task number is small, and we cannot integrate it into non-linear models. In this paper, we propose a two-step suboptimal unitary method (SUM) to combine a meta-learning strategy into multi-task models. In the first step, it searches for a global pattern by optimising the general parameters with gradient descents under constraints, which is a geological regularizer to enable model learning with less training data. In the second step, we derive an optimised model on each specific task from the global pattern with only a few local training data. Compared with traditional multi-task learning methods, SUM shows advantages of generalisation ability on distant tasks. It can be applied on any multi-task models with the gradient descent as its optimiser regardless if the prediction function is linear or not. Moreover, we can harness the model to enable traditional prediction model to make coKriging. The experiments on public datasets have suggested that our framework, when combined with current multi-task models, has a conspicuously better prediction result when the task number is small compared to low-rank tensor learning, and our model has a quite satisfying outcome when adjusting the current prediction models for coKriging.