Estimating the Jacobian matrix of an unknown multivariate function from sample values by means of a neural network

TL;DR

This paper introduces a neural network-based method to estimate the Jacobian matrix of an unknown multivariate function using sample data, with theoretical error bounds and practical implementation details.

Contribution

It presents a novel training approach that estimates Jacobians from sample pairs without explicit Jacobian data, including formal error bounds.

Findings

Provides an upper bound on the estimation error in operator norm.

Demonstrates the method's effectiveness through implementation and testing.

Bridges black-box function approximation and structural function analysis.

Abstract

We describe, implement and test a novel method for training neural networks to estimate the Jacobian matrix $J$ of an unknown multivariate function $F$. The training set is constructed from finitely many pairs $(x,F(x))$ and it contains no explicit information about $J$. The loss function for backpropagation is based on linear approximations and on a nearest neighbor search in the sample data. We formally establish an upper bound on the uniform norm of the error, in operator norm, between the estimated Jacobian matrix provided by the algorithm and the actual Jacobian matrix, under natural assumptions on the function, on the training set and on the loss of the neural network during training. The Jacobian matrix of a multivariate function contains a wealth of information about the function and it has numerous applications in science and engineering. The method given here represents a step in moving from black-box approximations of functions by neural networks to approximations that provide some structural information about the function in question.