# Assessing Random Forest self-reproducibility for optimal short biomarker signature discovery

## TL;DR

This paper introduces new metrics to help choose the best Random Forest algorithm for finding short, stable biomarker signatures in diagnostic tools.

## Contribution

The paper introduces two new metrics, HRS and HSS, to assess the stability of Random Forest algorithms for biomarker discovery.

## Key findings

- No single Random Forest implementation is universally suitable for all datasets and classification tasks.
- The choice of algorithm should be based on average AUC performance and AUC-derived stability metrics.
- Assessing hyper-stability scores improves confidence in selecting the best classification algorithm.

## Abstract

Biomarker signature discovery remains the main path to developing clinical diagnostic tools when the biological knowledge on pathology is weak. Shortest signatures are often preferred to reduce the cost of the diagnostic. The ability to find the best and shortest signature relies on the robustness of the models that can be built on such a set of molecules. The classification algorithm that will be used is often selected based on the average Area Under the Curve (AUC) performance of its models. However, it is not guaranteed that an algorithm with a large AUC distribution will keep a stable performance when facing data. Here, we propose two AUC-derived hyper-stability scores, the Hyper-stability Resampling Sensitive (HRS) and the Hyper-stability Signature Sensitive (HSS), as complementary metrics to the average AUC that should bring confidence in the choice for the best classification algorithm. To emphasize the importance of these scores, we compared 15 different Random Forest implementations. Our findings show that the Random Forest implementation should be chosen according to the data at hand and the classification question being evaluated. No Random Forest implementation can be used universally for any classification and on any dataset. Each of them should be tested for their average AUC performance and AUC-derived stability, prior to analysis.

Graphical AbstractGraphical abstract illustrating the study workflow for discovering robust models of short biomarker signatures using Random Forest algorithms. Assessing the AUC-derived stability (and hyper-stability) of Random Forest implementations is a key part of this process.