Randomness control and reproducibility study of random forest algorithm in R and Python

TL;DR

This study examines the reproducibility of random forest algorithms across R and Python implementations, focusing on controlling randomness to ensure consistent results in toxicological assessments.

Contribution

It compares four random forest packages and identifies key parameters and randomness sources affecting reproducibility across implementations.

Findings

Reproducibility depends on consistent PRNG and parameter settings.

Differences in implementations can lead to variability in results.

Guidelines for ensuring reproducibility across R and Python are proposed.

Abstract

When it comes to the safety of cosmetic products, compliance with regulatory standards is crucialto guarantee consumer protection against the risks of skin irritation. Toxicologists must thereforebe fully conversant with all risks. This applies not only to their day-to-day work, but also to allthe algorithms they integrate into their routines. Recognizing this, ensuring the reproducibility ofalgorithms becomes one of the most crucial aspects to address.However, how can we prove the robustness of an algorithm such as the random forest, that reliesheavily on randomness? In this report, we will discuss the strategy of integrating random forest intoocular tolerance assessment for toxicologists.We will compare four packages: randomForest and Ranger (R packages), adapted in Python via theSKRanger package, and the widely used Scikit-Learn with the RandomForestClassifier() function.Our goal is to investigate the parameters and sources of randomness affecting the outcomes ofRandom Forest algorithms.By setting comparable parameters and using the same Pseudo-Random Number Generator (PRNG),we expect to reproduce results consistently across the various available implementations of therandom forest algorithm. Nevertheless, this exploration will unveil hidden layers of randomness andguide our understanding of the critical parameters necessary to ensure reproducibility across all fourimplementations of the random forest algorithm.