From Rattle to Roar: Optimizer Showdown for MambaStock on S&P 500

TL;DR

This paper compares various optimizers for the MambaStock model predicting S&P 500 returns, introduces Roaree, a new optimizer balancing speed and stability, and demonstrates its effectiveness.

Contribution

The paper introduces Roaree, a novel optimizer that combines the training speed of Lion with reduced oscillations, improving forecasting performance.

Findings

Adam and RMSProp achieve lowest test errors.

Lion optimizer trains faster but with oscillations.

Roaree balances speed and stability effectively.

Abstract

We evaluate the performance of several optimizers on the task of forecasting S&P 500 Index returns with the MambaStock model. Among the most widely used algorithms, gradient-smoothing and adaptive-rate optimizers (for example, Adam and RMSProp) yield the lowest test errors. In contrast, the Lion optimizer offers notably faster training. To combine these advantages, we introduce a novel family of optimizers, Roaree, that dampens the oscillatory loss behavior often seen with Lion while preserving its training speed.