Unsupervised Ensemble Learning via Ising Model Approximation with Application to Phenotyping Prediction

TL;DR

This paper introduces unElisa, an unsupervised ensemble learning method that uses Ising model approximation to identify relevant classifiers and improve prediction accuracy, demonstrated on healthcare data for rheumatoid arthritis phenotyping.

Contribution

The paper proposes a novel unsupervised ensemble learning approach using Ising model approximation, including a pruning step and an augmented voting scheme, with theoretical guarantees and practical validation.

Findings

Successfully recovers true classifier neighborhood with exponential error decay

Achieves consistent Bayes classifier estimation in high-dimensional settings

Demonstrates improved phenotyping prediction on EHR data for rheumatoid arthritis

Abstract

Unsupervised ensemble learning has long been an interesting yet challenging problem that comes to prominence in recent years with the increasing demand of crowdsourcing in various applications. In this paper, we propose a novel method-- unsupervised ensemble learning via Ising model approximation (unElisa) that combines a pruning step with a predicting step. We focus on the binary case and use an Ising model to characterize interactions between the ensemble and the underlying true classifier. The presence of an edge between an observed classifier and the true classifier indicates a direct dependence whereas the absence indicates the corresponding one provides no additional information and shall be eliminated. This observation leads to the pruning step where the key is to recover the neighborhood of the true classifier. We show that it can be recovered successfully with exponentially decaying error in the high-dimensional setting by performing nodewise $\ell_1$-regularized logistic regression. The pruned ensemble allows us to get a consistent estimate of the Bayes classifier for predicting. We also propose an augmented version of majority voting by reversing all labels given by a subgroup of the pruned ensemble. We demonstrate the efficacy of our method through extensive numerical experiments and through the application to EHR-based phenotyping prediction on Rheumatoid Arthritis (RA) using data from Partners Healthcare System.