bioLeak: Leakage-Aware Modeling and Diagnostics for Machine Learning in R

TL;DR

bioLeak is an R package designed to detect and mitigate data leakage in biomedical machine learning, ensuring more reliable model evaluation and interpretation.

Contribution

It introduces leakage-aware resampling workflows and diagnostic tools tailored for complex biomedical data with repeated measures and heterogeneity.

Findings

Simulation shows performance varies with leakage mechanisms.

Case study demonstrates different conclusions with leaky vs. guarded pipelines.

Software supports multiple analysis tasks with comprehensive diagnostics.

Abstract

Data leakage remains a recurrent source of optimistic bias in biomedical machine learning studies. Standard row-wise cross-validation and globally estimated preprocessing steps are often inappropriate for data with repeated measurements, study-level heterogeneity, batch effects, or temporal dependencies. This paper describes bioLeak, an R package for constructing leakage-aware resampling workflows and for auditing fitted models for common leakage mechanisms. The package provides leakage-aware split construction, train-fold-only preprocessing, cross-validated model fitting, nested hyperparameter tuning, post hoc leakage audits, and HTML reporting. The implementation supports binary classification, multiclass classification, regression, and survival analysis, with task-specific metrics and S4 containers for splits, fits, audits, and inflation summaries. The simulation artifacts show how apparent performance changes under controlled leakage mechanisms, and the case study illustrates how guarded and leaky pipelines can yield materially different conclusions on multi-study transcriptomic data. The emphasis throughout is on software design, reproducible workflows, and interpretation of diagnostic output.