Bayesian Hierarchical Models for Quantitative Estimates for Performance metrics applied to Saddle Search Algorithms

TL;DR

This paper introduces a Bayesian hierarchical modeling framework for performance evaluation of saddle search algorithms in computational chemistry, providing nuanced insights into robustness and reliability across diverse molecular systems.

Contribution

It presents a novel Bayesian hierarchical approach for quantifying performance metrics and uncertainties, enabling more informed comparisons of algorithmic strategies in computational chemistry.

Findings

Conjugate Gradient (CG) is more robust than L-BFGS.

Removing external rotations increases computational cost but may improve L-BFGS reliability.

Supports adaptive workflows over single-method rankings.

Abstract

Rigorous performance evaluation is essential for developing robust algorithms for high-throughput computational chemistry. Traditional benchmarking, however, often struggles to account for system-specific variability, making it difficult to form actionable conclusions. We present a Bayesian hierarchical modeling framework that rigorously quantifies performance metrics and their uncertainty, enabling a nuanced comparison of algorithmic strategies. We apply this framework to analyze the Dimer method, comparing Conjugate Gradient (CG) and L-BFGS rotation optimizers, with and without the removal of external rotations, across a benchmark of 500 molecular systems. Our analysis confirms that CG offers higher overall robustness than L-BFGS in this context. While the theoretically-motivated removal of external rotations led to higher computational cost (>40% more energy and force calls) for most systems in this set, our models also reveal a subtle interplay, hinting that this feature may improve the reliability of the L-BFGS optimizer. Rather than identifying a single superior method, our findings support the design of adaptive "chain of methods" workflows. This work showcases how a robust statistical paradigm can move beyond simple performance rankings to inform the intelligent, context-dependent application of computational chemistry methods.