BenchML: an extensible pipelining framework for benchmarking representations of materials and molecules at scale

TL;DR

BenchML is a scalable, extensible framework for benchmarking diverse chemical representations using simple models, enabling unbiased evaluation and comparison of their effectiveness across datasets.

Contribution

Introduces a flexible ML benchmarking framework that evaluates raw chemical representations with simple models, promoting unbiased comparison and insight into their relative performance.

Findings

Topological and geometric representations show distinct strengths.

Benchmarking reveals the ease of learning different datasets.

Framework facilitates future method development and comparison.

Abstract

We introduce a machine-learning (ML) framework for high-throughput benchmarking of diverse representations of chemical systems against datasets of materials and molecules. The guiding principle underlying the benchmarking approach is to evaluate raw descriptor performance by limiting model complexity to simple regression schemes while enforcing best ML practices, allowing for unbiased hyperparameter optimization, and assessing learning progress through learning curves along series of synchronized train-test splits. The resulting models are intended as baselines that can inform future method development, next to indicating how easily a given dataset can be learnt. Through a comparative analysis of the training outcome across a diverse set of physicochemical, topological and geometric representations, we glean insight into the relative merits of these representations as well as their interrelatedness.