Minimizing robust density power-based divergences for general parametric density models

TL;DR

This paper introduces a stochastic method to efficiently minimize density power divergence for general parametric models, enhancing robustness against outliers and broadening applicability beyond specific distributions.

Contribution

It proposes a novel stochastic optimization approach for DPD that works with unnormalized models, overcoming computational challenges in general parametric density estimation.

Findings

Enables practical robust estimation for a wide range of models.

Provides an R package for implementation.

Improves computational efficiency over traditional methods.

Abstract

Density power divergence (DPD) is designed to robustly estimate the underlying distribution of observations, in the presence of outliers. However, DPD involves an integral of the power of the parametric density models to be estimated; the explicit form of the integral term can be derived only for specific densities, such as normal and exponential densities. While we may perform a numerical integration for each iteration of the optimization algorithms, the computational complexity has hindered the practical application of DPD-based estimation to more general parametric densities. To address the issue, this study introduces a stochastic approach to minimize DPD for general parametric density models. The proposed approach can also be employed to minimize other density power-based $\gamma$-divergences, by leveraging unnormalized models. We provide \verb|R| package for implementation of the proposed approach in \url{https://github.com/oknakfm/sgdpd}.