Learning Densities Conditional on Many Interacting Features

TL;DR

This paper introduces a nonparametric Bayesian method for modeling complex conditional densities based on many interacting features, utilizing tensor factorization and multistage feature selection for flexible, high-dimensional density estimation.

Contribution

It proposes a novel tensor factorization-based Bayesian approach for conditional density estimation that handles high-dimensional, interacting features with multistage feature selection.

Findings

The method effectively models complex, high-dimensional conditional densities.

It outperforms traditional techniques in capturing feature interactions.

The approach provides flexible density estimates that adapt to feature changes.

Abstract

Learning a distribution conditional on a set of discrete-valued features is a commonly encountered task. This becomes more challenging with a high-dimensional feature set when there is the possibility of interaction between the features. In addition, many frequently applied techniques consider only prediction of the mean, but the complete conditional density is needed to answer more complex questions. We demonstrate a novel nonparametric Bayes method based upon a tensor factorization of feature-dependent weights for Gaussian kernels. The method makes use of multistage feature selection for dimension reduction. The resulting conditional density morphs flexibly with the selected features.