On Selection Criteria for the Tuning Parameter in Robust Divergence

TL;DR

This paper introduces a new criterion for selecting the tuning parameter in robust divergence methods, improving inference efficiency by using an asymptotic Hyvarinen score approximation that is easy to compute.

Contribution

The paper proposes a novel selection criterion based on the Hyvarinen score for robust divergence, enhancing parameter tuning in robust statistical inference.

Findings

The criterion performs well in numerical studies with normal distributions.

It improves robustness and efficiency in regularized linear regression.

The method is computationally simple, requiring only derivatives of the density.

Abstract

While robust divergence such as density power divergence and $\gamma$-divergence is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnormalized model defined by robust divergence. The proposed selection criterion only requires first and second-order partial derivatives of an assumed density function with respect to observations, which can be easily computed regardless of the number of parameters. We demonstrate the usefulness of the proposed method via numerical studies using normal distributions and regularized linear regression.