Trainable Adaptive Activation Function Structure (TAAFS) Enhances Neural Network Force Field Performance with Only Dozens of Additional Parameters

TL;DR

This paper introduces TAAFS, a method that improves neural network force field accuracy by adaptively selecting activation functions with minimal additional parameters, validated through molecular dynamics simulations.

Contribution

The paper presents TAAFS, a novel approach that enhances NNFF performance by adaptively choosing activation functions without significantly increasing model complexity.

Findings

Accuracy improvements in neural network force fields

Effective integration of TAAFS across various models

Validation through molecular dynamics simulations

Abstract

At the heart of neural network force fields (NNFFs) is the architecture of neural networks, where the capacity to model complex interactions is typically enhanced through widening or deepening multilayer perceptrons (MLPs) or by increasing layers of graph neural networks (GNNs). These enhancements, while improving the model's performance, often come at the cost of a substantial increase in the number of parameters. By applying the Trainable Adaptive Activation Function Structure (TAAFS), we introduce a method that selects distinct mathematical formulations for non-linear activations, thereby increasing the precision of NNFFs with an insignificant addition to the parameter count. In this study, we integrate TAAFS into a variety of neural network models, resulting in observed accuracy improvements, and further validate these enhancements through molecular dynamics (MD) simulations using DeepMD.