Projection Pursuit Density Ratio Estimation

TL;DR

This paper introduces a novel projection pursuit-based method for density ratio estimation that effectively handles high-dimensional data, offering improved accuracy and consistency over existing techniques.

Contribution

The paper proposes a new PP-based approach for DRE that reduces dimensionality issues while maintaining model flexibility, with proven consistency and convergence.

Findings

Outperforms existing DRE methods in experiments

Demonstrates consistency and convergence of the estimator

Effectively mitigates high-dimensional challenges

Abstract

Density ratio estimation (DRE) is a paramount task in machine learning, for its broad applications across multiple domains, such as covariate shift adaptation, causal inference, independence tests and beyond. Parametric methods for estimating the density ratio possibly lead to biased results if models are misspecified, while conventional non-parametric methods suffer from the curse of dimensionality when the dimension of data is large. To address these challenges, in this paper, we propose a novel approach for DRE based on the projection pursuit (PP) approximation. The proposed method leverages PP to mitigate the impact of high dimensionality while retaining the model flexibility needed for the accuracy of DRE. We establish the consistency and the convergence rate for the proposed estimator. Experimental results demonstrate that our proposed method outperforms existing alternatives in various applications.