Nonconvex Extension of Generalized Huber Loss for Robust Learning and Pseudo-Mode Statistics

TL;DR

This paper introduces a nonconvex extension of the generalized Huber loss using a log-exp transform and logistic function, enabling robust learning with efficient convergence algorithms.

Contribution

It presents a novel nonconvex loss formulation that combines convexity and robustness, along with algorithms for fast convergence to minimizers.

Findings

The proposed loss combines properties of convex and robust loss functions.

A linear convergence algorithm is developed for minimization.

A derivative-free exponential convergence rate algorithm is introduced.

Abstract

We propose an extended generalization of the pseudo Huber loss formulation. We show that using the log-exp transform together with the logistic function, we can create a loss which combines the desirable properties of the strictly convex losses with robust loss functions. With this formulation, we show that a linear convergence algorithm can be utilized to find a minimizer. We further discuss the creation of a quasi-convex composite loss and provide a derivative-free exponential convergence rate algorithm.