Sequential double cross-validation for assessment of added predictive ability in high-dimensional omic applications

TL;DR

This paper introduces a sequential double cross-validation method combined with regularized regression to evaluate the incremental predictive value of new omic data sources in high-dimensional settings, demonstrated on BMI prediction.

Contribution

The paper presents a novel two-step validation approach with performance indices and permutation testing for assessing added predictive ability of omic datasets.

Findings

Metabolomics and transcriptomics data improve BMI prediction.

The method effectively quantifies added predictive value.

Simulation studies confirm the test's validity.

Abstract

Enriching existing predictive models with new biomolecular markers is an important task in the new multi-omic era. Clinical studies increasingly include new sets of omic measurements which may prove their added value in terms of predictive performance. We introduce a two-step approach for the assessment of the added predictive ability of omic predictors, based on sequential double cross-validation and regularized regression models. We propose several performance indices to summarize the two-stage prediction procedure and a permutation test to formally assess the added predictive value of a second omic set of predictors over a primary omic source. The performance of the test is investigated through simulations. We illustrate the new method through the systematic assessment and comparison of the performance of transcriptomics and metabolomics sources in the prediction of body mass index (BMI) using longitudinal data from the Dietary, Lifestyle, and Genetic determinants of Obesity and Metabolic syndrome (DILGOM) study, a population-based cohort from Finland.