Provably Robust Pre-Trained Ensembles for Biomarker-Based Cancer Classification

TL;DR

This paper introduces a robust ensemble approach using pre-trained Hyperfast models for cancer classification, achieving high accuracy and resilience to class imbalance with fewer features and minimal tuning.

Contribution

It presents a novel ensemble combining Hyperfast, XGBoost, and LightGBM for multi-class cancer classification, demonstrating improved accuracy and robustness over existing methods.

Findings

Achieved highest AUC of 0.9929 on binary cancer classification.

Demonstrated robustness under class imbalance with balanced accuracy and recall.

Reduced feature usage to 500 PCA components while maintaining high accuracy.

Abstract

Certain cancer types, notably pancreatic cancer, are difficult to detect at an early stage, motivating robust biomarker-based screening. Liquid biopsies enable non-invasive monitoring of circulating biomarkers, but typical machine learning pipelines for high-dimensional tabular data (e.g., random forests, SVMs) rely on expensive hyperparameter tuning and can be brittle under class imbalance. We leverage a meta-trained Hyperfast model for classifying cancer, accomplishing the highest AUC of 0.9929 and simultaneously achieving robustness especially on highly imbalanced datasets compared to other ML algorithms in several binary classification tasks (e.g. breast invasive carcinoma; BRCA vs. non-BRCA). We also propose a novel ensemble model combining pre-trained Hyperfast model, XGBoost, and LightGBM for multi-class classification tasks, achieving an incremental increase in accuracy (0.9464) while merely using 500 PCA features; distinguishable from previous studies where they used more than 2,000 features for similar results. Crucially, we demonstrate robustness under class imbalance: empirically via balanced accuracy and minority-class recall across cancer-vs.-noncancer and cancer-vs.-rest settings, and theoretically by showing (i) a prototype-form final layer for Hyperfast that yields prior-insensitive decisions under bounded bias, and (ii) minority-error reductions for majority vote under mild error diversity. Together, these results indicate that pre-trained tabular models and simple ensembling can deliver state-of-the-art accuracy and improved minority-class performance with far fewer features and no additional tuning.