# Time-Smoothed Gradients for Online Forecasting

## TL;DR

This paper introduces a time-smoothed gradient method for online forecasting with SGD, reducing sensitivity to learning rate tuning and providing more stable, computationally efficient results.

## Contribution

It proposes a novel time-smoothed gradient update rule for SGD in online forecasting, addressing learning rate sensitivity and stability issues.

## Key findings

- More stable forecasting results on GEFCom2014 dataset
- Approach is computationally efficient
- Outperforms existing methods in stability

## Abstract

Here, we study different update rules in stochastic gradient descent (SGD) for online forecasting problems. The selection of the learning rate parameter is critical in SGD. However, it may not be feasible to tune this parameter in online learning. Therefore, it is necessary to have an update rule that is not sensitive to the selection of the learning parameter. Inspired by the local regret metric that we introduced previously, we propose to use time-smoothed gradients within SGD update. Using the public data set-- GEFCom2014, we validate that our approach yields more stable results than the other existing approaches. Furthermore, we show that such a simple approach is computationally efficient compared to the alternatives.