ZetA: A Riemann Zeta-Scaled Extension of Adam for Deep Learning

TL;DR

ZetA introduces a novel optimizer for deep learning that leverages the Riemann zeta function for dynamic gradient scaling, enhancing generalization and robustness over Adam in various image classification tasks.

Contribution

It is the first optimizer to incorporate zeta-based gradient scaling, combining multiple techniques to improve deep learning optimization.

Findings

Consistent accuracy improvements over Adam on multiple datasets.

Effective in noisy and high-granularity classification tasks.

Computationally efficient and robust in experiments.

Abstract

This work introduces ZetA, a novel deep learning optimizer that extends Adam by incorporating dynamic scaling based on the Riemann zeta function. To the best of our knowledge, ZetA is the first optimizer to apply zeta-based gradient scaling within deep learning optimization. The method improves generalization and robustness through a hybrid update mechanism that integrates adaptive damping, cosine similarity-based momentum boosting, entropy-regularized loss, and Sharpness-Aware Minimization (SAM)-style perturbations. Empirical evaluations on SVHN, CIFAR10, CIFAR100, STL10, and noisy CIFAR10 consistently show test accuracy improvements over Adam. All experiments employ a lightweight fully connected network trained for five epochs under mixed-precision settings. The results demonstrate that ZetA is a computationally efficient and robust alternative to Adam, particularly effective in noisy or high-granularity classification tasks.