A Dependent Feature Allocation Model Based on Random Fields

TL;DR

This paper presents a novel dependent feature allocation model using Gaussian Markov Random Fields to incorporate covariates and complex dependence structures, demonstrated on a polypharmacy dataset.

Contribution

It introduces a flexible GMRF-based framework for dependent feature allocations that handles high-dimensional data with complex dependence structures and dynamic extensions.

Findings

Successfully inferred latent health conditions from patient drug profiles.

Effectively modeled covariate-dependent feature allocations with complex dependence.

Extended to dynamic spatio-temporal processes for evolving item effects.

Abstract

We introduce a flexible framework for modeling dependent feature allocations. Our approach addresses limitations in traditional nonparametric methods by directly modeling the logit-probability surface of the feature paintbox, enabling the explicit incorporation of covariates and complex but tractable dependence structures. The core of our model is a Gaussian Markov Random Field (GMRF), which we use to robustly decompose the latent field, separating a structural component based on the baseline covariates from intrinsic, unstructured heterogeneity. This structure is not a rigid grid but a sparse k-nearest neighbors graph derived from the latent geometry in the data, ensuring high-dimensional tractability. We extend this framework to a dynamic spatio-temporal process, allowing item effects to evolve via an Ornstein-Uhlenbeck process. Feature correlations are captured using a low-rank factorization of their joint prior. We demonstrate our model's utility by applying it to a polypharmacy dataset, successfully inferring latent health conditions from patient drug profiles.