chemmodlab: A Cheminformatics Modeling Laboratory for Fitting and Assessing Machine Learning Models

TL;DR

chemmodlab is an R package designed to simplify the process of fitting, assessing, and comparing machine learning models in cheminformatics, with broad applicability for the machine learning community.

Contribution

it introduces a user-friendly framework for model evaluation and comparison, including novel statistical significance visualization methods for machine learning performance.

Findings

Provides tools for model fitting and assessment in R

Includes a novel multiple comparisons similarity plot

Uses repeated k-fold cross validation with adjustments

Abstract

The goal of chemmodlab is to streamline the fitting and assessment pipeline for many machine learning models in R, making it easy for researchers to compare the utility of new models. While focused on implementing methods for model fitting and assessment that have been accepted by experts in the cheminformatics field, all of the methods in chemmodlab have broad utility for the machine learning community. chemmodlab contains several assessment utilities including a plotting function that constructs accumulation curves and a function that computes many performance measures. The most novel feature of chemmodlab is the ease with which statistically significant performance differences for many machine learning models is presented by means of the multiple comparisons similarity plot. Differences are assessed using repeated k-fold cross validation where blocking increases precision and multiplicity adjustments are applied.