Two-sample comparison through additive tree models for density ratios

TL;DR

This paper introduces additive tree models with a novel balancing loss for two-sample density ratio estimation, enabling efficient, Bayesian uncertainty quantification and applications to microbiome data.

Contribution

It proposes a new loss function and Bayesian framework for density ratio estimation using tree models, enhancing accuracy, efficiency, and uncertainty quantification.

Findings

Achieves accurate density ratio estimation with computational efficiency.

Provides Bayesian uncertainty quantification for high-dimensional data.

Demonstrates application to microbiome data quality assessment.

Abstract

The ratio of two densities provides a direct characterization of their differences. We consider the two-sample comparison problem by estimating this ratio given i.i.d. observations from two distributions. To this end, we propose additive tree models for density ratio estimation along with efficient algorithms using a new loss function, the balancing loss. The loss allows tree-based models to be trained using several algorithms originally designed for supervised learning, such as forward-stagewise optimization and gradient boosting. Moreover, the balancing loss resembles an exponential family kernel, and it can serve as a pseudo-likelihood with conjugate priors. This property enables generalized Bayesian inference on the density ratio using backfitting samplers designed for Bayesian additive regression trees (BART). Our Bayesian strategy provides uncertainty quantification for the inferred density ratio, which is critical for applications involving high-dimensional and data-limited distributions with potentially substantial uncertainty. We further show connections of the balancing loss to the exponential loss in binary classification and to the variational form of f-divergence, particularly the squared Hellinger distance. Numerical experiments demonstrate that our method achieves both accuracy and computational efficiency, while uniquely providing uncertainty quantification. Finally, we demonstrate its application to assessing the quality of generative models for microbiome compositional data.