Non-collective Calibrating Strategy for Time Series Forecasting

TL;DR

This paper introduces a universal calibrating strategy called Socket+Plug (SoP) that enhances existing deep learning time series forecasting models with minimal additional training, leading to significant performance improvements.

Contribution

The paper proposes the novel Socket+Plug (SoP) calibrating strategy that is model-agnostic and improves existing models without retraining from scratch.

Findings

Up to 22% performance improvement on benchmarks.

Effective calibration across various deep forecasting models.

Applicable to simple and complex architectures alike.

Abstract

Deep learning-based approaches have demonstrated significant advancements in time series forecasting. Despite these ongoing developments, the complex dynamics of time series make it challenging to establish the rule of thumb for designing the golden model architecture. In this study, we argue that refining existing advanced models through a universal calibrating strategy can deliver substantial benefits with minimal resource costs, as opposed to elaborating and training a new model from scratch. We first identify a multi-target learning conflict in the calibrating process, which arises when optimizing variables across time steps, leading to the underutilization of the model's learning capabilities. To address this issue, we propose an innovative calibrating strategy called Socket+Plug (SoP). This approach retains an exclusive optimizer and early-stopping monitor for each predicted target within each Plug while keeping the fully trained Socket backbone frozen. The model-agnostic nature of SoP allows it to directly calibrate the performance of any trained deep forecasting models, regardless of their specific architectures. Extensive experiments on various time series benchmarks and a spatio-temporal meteorological ERA5 dataset demonstrate the effectiveness of SoP, achieving up to a 22% improvement even when employing a simple MLP as the Plug (highlighted in Figure 1). Code is available at https://github.com/hanyuki23/SoP.