A Reproducible Framework for Bias-Resistant Machine Learning on Small-Sample Neuroimaging Data

TL;DR

This paper presents a reproducible machine learning framework tailored for small-sample neuroimaging data, emphasizing bias resistance, interpretability, and unbiased evaluation to improve reproducibility and generalization.

Contribution

It introduces a novel framework combining domain-informed feature engineering, nested cross-validation, and calibrated decision thresholds for small-sample neuroimaging studies.

Findings

Achieved a nested-CV balanced accuracy of 0.660 with MRI data.

Selected a compact, interpretable feature subset.

Demonstrated improved reproducibility and unbiased evaluation.

Abstract

We introduce a reproducible, bias-resistant machine learning framework that integrates domain-informed feature engineering, nested cross-validation, and calibrated decision-threshold optimization for small-sample neuroimaging data. Conventional cross-validation frameworks that reuse the same folds for both model selection and performance estimation yield optimistically biased results, limiting reproducibility and generalization. Demonstrated on a high-dimensional structural MRI dataset of deep brain stimulation cognitive outcomes, the framework achieved a nested-CV balanced accuracy of 0.660\,$\pm$\,0.068 using a compact, interpretable subset selected via importance-guided ranking. By combining interpretability and unbiased evaluation, this work provides a generalizable computational blueprint for reliable machine learning in data-limited biomedical domains.