On Difference Between Two Types of $\gamma$-divergence for Regression

TL;DR

This paper compares two types of $ ext{gamma}$-divergence for regression, highlighting their differences in robustness under heterogeneous contamination and extending previous results to general parametric models.

Contribution

It identifies a key difference in robustness between the two divergences under specific contamination conditions and generalizes prior findings without restrictive assumptions.

Findings

One divergence maintains strong robustness under heterogeneous contamination.

The other divergence is robust only under certain model conditions or homogeneous contamination.

The results apply broadly to any parametric regression model, not just logistic regression.

Abstract

The $\gamma$-divergence is well-known for having strong robustness against heavy contamination. By virtue of this property, many applications via the $\gamma$-divergence have been proposed. There are two types of \gd\ for regression problem, in which the treatments of base measure are different. In this paper, we compare them and pointed out a distinct difference between these two divergences under heterogeneous contamination where the outlier ratio depends on the explanatory variable. One divergence has the strong robustness under heterogeneous contamination. The other does not have in general, but has when the parametric model of the response variable belongs to a location-scale family in which the scale does not depend on the explanatory variables or under homogeneous contamination where the outlier ratio does not depend on the explanatory variable. \citet{hung.etal.2017} discussed the strong robustness in a logistic regression model with an additional assumption that the tuning parameter $\gamma$ is sufficiently large. The results obtained in this paper hold for any parametric model without such an additional assumption.