On Robustness of the Normalized Subgradient Method with Randomly Corrupted Subgradients

TL;DR

This paper investigates the robustness of the normalized subgradient method when subgradient information is arbitrarily corrupted, demonstrating convergence under probabilistic corruption for various convex functions and supporting results with numerical experiments.

Contribution

It provides the first analysis of the normalized subgradient method's robustness to random subgradient corruption, proving convergence for multiple convexity settings.

Findings

Converges to a minimizer despite subgradient corruption

Applicable to convex, strongly convex, and weakly-pseudo convex functions

Numerical experiments confirm theoretical results

Abstract

Numerous modern optimization and machine learning algorithms rely on subgradient information being trustworthy and hence, they may fail to converge when such information is corrupted. In this paper, we consider the setting where subgradient information may be arbitrarily corrupted (with a given probability) and study the robustness properties of the normalized subgradient method. Under the probabilistic corruption scenario, we prove that the normalized subgradient method, whose updates rely solely on directional information of the subgradient, converges to a minimizer for convex, strongly convex, and weakly-pseudo convex functions satisfying certain conditions. Numerical evidence on linear regression and logistic classification problems support our results.