Automated Machine Learning in Radiomics: A Comparative Evaluation of Performance, Efficiency and Accessibility

TL;DR

This study compares general-purpose and radiomics-specific AutoML frameworks on diverse radiomics tasks, highlighting strengths and gaps in performance, efficiency, and accessibility, and suggesting directions for future development.

Contribution

It provides a comprehensive evaluation of AutoML tools in radiomics, revealing that general-purpose frameworks are more accessible while radiomics-specific tools need improvements in performance and usability.

Findings

Simplatab achieved highest average AUC (81.81%) with moderate runtime (~1 hour)

LightAutoML showed fastest execution with 78.74% mean AUC in six minutes

Most radiomics-specific frameworks were obsolete or computationally inefficient

Abstract

Automated machine learning (AutoML) frameworks can lower technical barriers for predictive and prognostic model development in radiomics by enabling researchers without programming expertise to build models. However, their effectiveness in addressing radiomics-specific challenges remains unclear. This study evaluates the performance, efficiency, and accessibility of general-purpose and radiomics-specific AutoML frameworks on diverse radiomics classification tasks, thereby highlighting development needs for radiomics. Ten public/private radiomics datasets with varied imaging modalities (CT/MRI), sizes, anatomies and endpoints were used. Six general-purpose and five radiomics-specific frameworks were tested with predefined parameters using standardized cross-validation. Evaluation metrics included AUC, runtime, together with qualitative aspects related to software status, accessibility, and interpretability. Simplatab, a radiomics-specific tool with a no-code interface, achieved the highest average test AUC (81.81%) with a moderate runtime (~1 hour). LightAutoML, a general-purpose framework, showed the fastest execution with competitive performance (78.74% mean AUC in six minutes). Most radiomics-specific frameworks were excluded from the performance analysis due to obsolescence, extensive programming requirements, or computational inefficiency. Conversely, general-purpose frameworks demonstrated higher accessibility and ease of implementation. Simplatab provides an effective balance of performance, efficiency, and accessibility for radiomics classification problems. However, significant gaps remain, including the lack of accessible survival analysis support and the limited integration of feature reproducibility and harmonization within current AutoML frameworks. Future research should focus on adapting AutoML solutions to better address these radiomics-specific challenges.