Robust Time Series Forecasting with Non-Heavy-Tailed Gaussian Loss-Weighted Sampler

TL;DR

This paper introduces a Gaussian loss-weighted sampler for multivariate time series forecasting that mitigates heavy-tailed loss issues, improves training efficiency, and enhances prediction accuracy by focusing on samples with average losses.

Contribution

The paper proposes a novel Gaussian loss-weighted sampling method that reduces overfitting to outliers and redundant easy samples in time series forecasting.

Findings

Improves prediction accuracy by 1%-4% in real-world datasets.

Reduces overfitting to outliers and easy samples.

Enhances training efficiency by focusing on average-loss samples.

Abstract

Forecasting multivariate time series is a computationally intensive task challenged by extreme or redundant samples. Recent resampling methods aim to increase training efficiency by reweighting samples based on their running losses. However, these methods do not solve the problems caused by heavy-tailed distribution losses, such as overfitting to outliers. To tackle these issues, we introduce a novel approach: a Gaussian loss-weighted sampler that multiplies their running losses with a Gaussian distribution weight. It reduces the probability of selecting samples with very low or very high losses while favoring those close to average losses. As it creates a weighted loss distribution that is not heavy-tailed theoretically, there are several advantages to highlight compared to existing methods: 1) it relieves the inefficiency in learning redundant easy samples and overfitting to outliers, 2) It improves training efficiency by preferentially learning samples close to the average loss. Application on real-world time series forecasting datasets demonstrate improvements in prediction quality for 1%-4% using mean square error measurements in channel-independent settings. The code will be available online after 1 the review.