Jacobian-Enhanced Neural Networks

TL;DR

Jacobian-Enhanced Neural Networks (JENN) improve surrogate modeling accuracy and efficiency by training neural networks to predict derivatives, benefiting optimization tasks in computationally intensive fields like design and physics-based modeling.

Contribution

This paper introduces JENN, a novel neural network training method that emphasizes accurate derivative prediction, enhancing surrogate model performance for optimization.

Findings

JENN outperforms standard neural networks in surrogate modeling accuracy.

JENN enables faster and more reliable surrogate-based optimization.

Theoretical foundations for JENN are fully derived and validated.

Abstract

Jacobian-Enhanced Neural Networks (JENN) are densely connected multi-layer perceptrons, whose training process is modified to predict partial derivatives accurately. Their main benefit is better accuracy with fewer training points compared to standard neural networks. These attributes are particularly desirable in the field of computer-aided design, where there is often the need to replace computationally expensive, physics-based models with fast running approximations, known as surrogate models or meta-models. Since a surrogate emulates the original model accurately in near-real time, it yields a speed benefit that can be used to carry out orders of magnitude more function calls quickly. However, in the special case of gradient-enhanced methods, there is the additional value proposition that partial derivatives are accurate, which is a critical property for one important use-case: surrogate-based optimization. This work derives the complete theory and exemplifies its superiority over standard neural nets for surrogate-based optimization.