Investigating the hyperparameter space of deep neural network models for reaction coordinates

TL;DR

This study explores how hyperparameter choices in deep neural networks influence the quality of reaction coordinate predictions, revealing that diverse models can describe the same underlying mechanisms with similar accuracy.

Contribution

Developed a Bayesian hyperparameter tuning approach for DNNs in reaction coordinate prediction and analyzed how hyperparameters affect model features and performance.

Findings

Diverse DNN structures can predict reaction coordinates with similar accuracy.

Hyperparameter space is multimodal, with different models sharing similar features.

Solvent electrostatic potential significantly influences the reaction coordinate in water.

Abstract

Identifying reaction coordinates (RCs) is a key to understanding the mechanism of reactions in complex systems. Deep neural network (DNN) and machine learning approaches have become a powerful tool to find the RC. On the other hand, the hyperparameters that determine the DNN model structure can be highly flexible and are often selected intuitively and in a non-trivial and tedious manner. Furthermore, how the hyperparameter choice affects the RC quality remains obscure. Here, we explore the hyperparameter space by developing the hyperparameter tuning approach for the DNN model for RC and investigate how the parameter set affects the RC quality. The DNN model is built to predict the committor along the RC from various collective variables by minimizing the cross-entropy function; the hyperparameters are automatically determined using the Bayesian optimization method. The approach is applied to study the isomerization of alanine dipeptide in vacuum and in water, and the features that characterize the RC are extracted using the explainable AI (XAI) tools. The results show that the DNN models with diverse structures can describe the RC with similar accuracy, and furthermore, the features analyzed by XAI are highly similar. This indicates that the hyperparameter space is multimodal. The electrostatic potential from the solvent to the hydrogen H18 plays an important role in the RC in water. The current study shows that the structure of the DNN models can be rather flexible, while the suitably optimized models share the same features; therefore, a common mechanism from the RC can be extracted.