LEARN: An Invex Loss for Outlier Oblivious Robust Online Optimization

TL;DR

This paper introduces LEARN, a novel invex loss function for robust online convex optimization that effectively handles outliers without Lipschitz assumptions, providing tight regret guarantees and a unified analysis framework.

Contribution

The paper proposes the LEARN loss and a robust online gradient descent variant, offering the first tight regret bounds for invex losses in adversarial outlier settings.

Findings

LEARN loss mitigates outlier effects effectively.

Robust online gradient descent achieves tight regret bounds.

Unified analysis framework for non-convex invex losses.

Abstract

We study a robust online convex optimization framework, where an adversary can introduce outliers by corrupting loss functions in an arbitrary number of rounds k, unknown to the learner. Our focus is on a novel setting allowing unbounded domains and large gradients for the losses without relying on a Lipschitz assumption. We introduce the Log Exponential Adjusted Robust and iNvex (LEARN) loss, a non-convex (invex) robust loss function to mitigate the effects of outliers and develop a robust variant of the online gradient descent algorithm by leveraging the LEARN loss. We establish tight regret guarantees (up to constants), in a dynamic setting, with respect to the uncorrupted rounds and conduct experiments to validate our theory. Furthermore, we present a unified analysis framework for developing online optimization algorithms for non-convex (invex) losses, utilizing it to provide regret bounds with respect to the LEARN loss, which may be of independent interest.