Gradient-enhanced deep neural network approximations

TL;DR

This paper introduces gradient-enhanced deep neural networks that incorporate gradient information into the training process, improving approximation accuracy and uncertainty quantification over traditional methods.

Contribution

It presents a novel approach that integrates gradient data into DNN training as regularization, with theoretical analysis and practical demonstrations of improved performance.

Findings

Outperforms traditional DNNs in approximation accuracy

Effective for gradient-enhanced uncertainty quantification

Theoretical posterior estimates similar to existing regularized DNNs

Abstract

We propose in this work the gradient-enhanced deep neural networks (DNNs) approach for function approximations and uncertainty quantification. More precisely, the proposed approach adopts both the function evaluations and the associated gradient information to yield enhanced approximation accuracy. In particular, the gradient information is included as a regularization term in the gradient-enhanced DNNs approach, for which we present similar posterior estimates (by the two-layer neural networks) as those in the path-norm regularized DNNs approximations. We also discuss the application of this approach to gradient-enhanced uncertainty quantification, and present several numerical experiments to show that the proposed approach can outperform the traditional DNNs approach in many cases of interests.