ADAM Optimization with Adaptive Batch Selection

TL;DR

This paper introduces AdamCB, an improved adaptive optimizer that uses combinatorial bandit sampling to enhance convergence and practical performance in neural network training.

Contribution

It integrates combinatorial bandit techniques into Adam, providing better theoretical guarantees and empirical performance over previous adaptive sampling methods.

Findings

AdamCB achieves faster convergence than previous Adam variants.

Numerical experiments show AdamCB consistently outperforms existing methods.

Theoretical analysis confirms improved regret bounds for AdamCB.

Abstract

Adam is a widely used optimizer in neural network training due to its adaptive learning rate. However, because different data samples influence model updates to varying degrees, treating them equally can lead to inefficient convergence. To address this, a prior work proposed adapting the sampling distribution using a bandit framework to select samples adaptively. While promising, the bandit-based variant of Adam suffers from limited theoretical guarantees. In this paper, we introduce Adam with Combinatorial Bandit Sampling (AdamCB), which integrates combinatorial bandit techniques into Adam to resolve these issues. AdamCB is able to fully utilize feedback from multiple samples at once, enhancing both theoretical guarantees and practical performance. Our regret analysis shows that AdamCB achieves faster convergence than Adam-based methods including the previous bandit-based variant. Numerical experiments demonstrate that AdamCB consistently outperforms existing methods.