Exact Soft Confidence-Weighted Learning

TL;DR

This paper introduces a Soft Confidence-Weighted learning algorithm that improves upon traditional confidence-weighted methods by effectively handling non-separable data, achieving better accuracy and efficiency in online learning tasks.

Contribution

The paper presents a novel Soft Confidence-Weighted learning scheme that extends confidence-weighted algorithms to non-separable data with adaptive margins and improved computational efficiency.

Findings

SCW outperforms original CW in accuracy and efficiency

SCW achieves comparable or better results than state-of-the-art algorithms

SCW requires fewer updates and less computation

Abstract

In this paper, we propose a new Soft Confidence-Weighted (SCW) online learning scheme, which enables the conventional confidence-weighted learning method to handle non-separable cases. Unlike the previous confidence-weighted learning algorithms, the proposed soft confidence-weighted learning method enjoys all the four salient properties: (i) large margin training, (ii) confidence weighting, (iii) capability to handle non-separable data, and (iv) adaptive margin. Our experimental results show that the proposed SCW algorithms significantly outperform the original CW algorithm. When comparing with a variety of state-of-the-art algorithms (including AROW, NAROW and NHERD), we found that SCW generally achieves better or at least comparable predictive accuracy, but enjoys significant advantage of computational efficiency (i.e., smaller number of updates and lower time cost).