Online Dual Coordinate Ascent Learning

TL;DR

This paper introduces an online dual coordinate-ascent algorithm that efficiently handles streaming data, enabling continuous adaptation and learning without revisiting past data, addressing limitations of traditional S-DCA methods.

Contribution

The paper develops an online dual coordinate-ascent algorithm suitable for streaming data, extending S-DCA to online scenarios with no need for multiple data passes.

Findings

Enables real-time learning from streaming data

Provides theoretical guarantees for online adaptation

Demonstrates effectiveness in online learning scenarios

Abstract

The stochastic dual coordinate-ascent (S-DCA) technique is a useful alternative to the traditional stochastic gradient-descent algorithm for solving large-scale optimization problems due to its scalability to large data sets and strong theoretical guarantees. However, the available S-DCA formulation is limited to finite sample sizes and relies on performing multiple passes over the same data. This formulation is not well-suited for online implementations where data keep streaming in. In this work, we develop an {\em online} dual coordinate-ascent (O-DCA) algorithm that is able to respond to streaming data and does not need to revisit the past data. This feature embeds the resulting construction with continuous adaptation, learning, and tracking abilities, which are particularly attractive for online learning scenarios.