Adaptive learning of density ratios in RKHS

TL;DR

This paper introduces an adaptive method for density ratio estimation in RKHS that achieves minimax optimal error rates without prior knowledge of the density ratio's regularity, supported by theoretical bounds and numerical results.

Contribution

It develops a new adaptive density ratio estimation method in RKHS with finite-sample error bounds and a Lepskii type parameter choice, achieving optimal rates.

Findings

Derived new finite-sample error bounds for the method.

Proposed a Lepskii type parameter choice principle.

Achieved minimax optimal error rate in the quadratic loss case.

Abstract

Estimating the ratio of two probability densities from finitely many observations of the densities is a central problem in machine learning and statistics with applications in two-sample testing, divergence estimation, generative modeling, covariate shift adaptation, conditional density estimation, and novelty detection. In this work, we analyze a large class of density ratio estimation methods that minimize a regularized Bregman divergence between the true density ratio and a model in a reproducing kernel Hilbert space (RKHS). We derive new finite-sample error bounds, and we propose a Lepskii type parameter choice principle that minimizes the bounds without knowledge of the regularity of the density ratio. In the special case of quadratic loss, our method adaptively achieves a minimax optimal error rate. A numerical illustration is provided.