Accelerated Randomized Coordinate Descent Algorithms for Stochastic Optimization and Online Learning

TL;DR

This paper introduces accelerated randomized coordinate descent algorithms that improve efficiency and regret performance in stochastic optimization and online learning, matching or surpassing existing methods.

Contribution

The paper presents novel accelerated randomized coordinate descent algorithms with lower per-iteration complexity and enhanced regret performance for online learning.

Findings

Reduced per-iteration complexity compared to accelerated gradient methods

Improved regret bounds in online learning scenarios

Competitive convergence rates in stochastic optimization

Abstract

We propose accelerated randomized coordinate descent algorithms for stochastic optimization and online learning. Our algorithms have significantly less per-iteration complexity than the known accelerated gradient algorithms. The proposed algorithms for online learning have better regret performance than the known randomized online coordinate descent algorithms. Furthermore, the proposed algorithms for stochastic optimization exhibit as good convergence rates as the best known randomized coordinate descent algorithms. We also show simulation results to demonstrate performance of the proposed algorithms.