Separating and reintegrating latent variables to improve classification of genomic data

TL;DR

This paper introduces the cross-residualization classifier, a novel method that estimates and adjusts for latent variables in genomic data, improving classification accuracy by effectively separating and reintegrating these hidden effects.

Contribution

The paper presents a new classifier that accounts for latent variables in genomic data by estimating, residualizing, and reintegrating them, enhancing classification performance.

Findings

Performs well on simulated data and real genomic datasets.

Offers substantial gains over existing classifiers.

Effectively separates and reintegrates latent variables.

Abstract

Genomic datasets contain the effects of various unobserved biological variables in addition to the variable of primary interest. These latent variables often affect a large number of features (e.g., genes) and thus give rise to dense latent variation, which presents both challenges and opportunities for classification. Some of these latent variables may be partially correlated with the phenotype of interest and therefore helpful, while others may be uncorrelated and thus merely contribute additional noise. Moreover, whether potentially helpful or not, these latent variables may obscure weaker effects that impact only a small number of features but more directly capture the signal of primary interest. We propose the cross-residualization classifier to better account for the latent variables in genomic data. Through an adjustment and ensemble procedure, the cross-residualization classifier essentially estimates the latent variables and residualizes out their effects, trains a classifier on the residuals, and then re-integrates the the latent variables in a final ensemble classifier. Thus, the latent variables are accounted for without discarding any potentially predictive information that they may contribute. We apply the method to simulated data as well as a variety of genomic datasets from multiple platforms. In general, we find that the cross-residualization classifier performs well relative to existing classifiers and sometimes offers substantial gains.