Normalized Online Learning

TL;DR

This paper presents online learning algorithms that adapt to feature scales, eliminating the need for pre-normalization and improving robustness and efficiency in test scenarios.

Contribution

The authors introduce scale-invariant online learning algorithms with regret bounds based on feature scale ratios, enhancing robustness and reducing preprocessing.

Findings

Algorithms are independent of feature scales.

Regret bounds depend on scale ratios, not absolute scales.

Reduced need for data normalization and improved robustness.

Abstract

We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale. This has several useful effects: there is no need to pre-normalize data, the test-time and test-space complexity are reduced, and the algorithms are more robust.