Hierarchical infinite factor model for improving the prediction of surgical complications for geriatric patients

TL;DR

This paper introduces a hierarchical infinite latent factor model with a novel prior to better capture covariance structures across subpopulations, significantly improving prediction of surgical complications in geriatric patients.

Contribution

The paper presents a new hierarchical infinite factor model with a Dirichlet process prior, enhancing predictive accuracy and inference in heterogeneous subpopulation data.

Findings

Improved sensitivity in predicting surgical complications to 91%.

Model effectively captures subpopulation covariance structures.

Outperforms baseline methods in simulations.

Abstract

We develop a hierarchical infinite latent factor model (HIFM) to appropriately account for the covariance structure across subpopulations in data. We propose a novel Hierarchical Dirichlet Process shrinkage prior on the loadings matrix that flexibly captures the underlying structure of our data across subpopulations while sharing information to improve inference and prediction. The stick-breaking construction of the prior assumes infinite number of factors and allows for each subpopulation to utilize different subsets of the factor space and select the number of factors needed to best explain the variation. Theoretical results are provided to show support of the prior. We develop the model into a latent factor regression method that excels at prediction and inference of regression coefficients. Simulations are used to validate this strong performance compared to baseline methods. We apply this work to the problem of predicting surgical complications using electronic health record data for geriatric patients at Duke University Health System (DUHS). We utilize additional surgical encounters at DUHS to enhance learning for the targeted patients. Using HIFM to identify high risk patients improves the sensitivity of predicting death to 91% from 35% based on the currently used heuristic.