On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networks

TL;DR

This paper analyzes the approximation capabilities of invertible neural networks (INNs) for bi-Lipschitz maps, providing theoretical bounds and a novel approach for infinite-dimensional spaces, with promising numerical results.

Contribution

It offers the first detailed analysis of the approximation rate of INNs for bi-Lipschitz maps and introduces a new method for infinite-dimensional approximation combining model reduction and INNs.

Findings

INNs can effectively approximate both forward and inverse bi-Lipschitz maps.

The proposed approach successfully approximates solution operators for parameterized elliptic problems.

Numerical experiments demonstrate the feasibility of the method in practical scenarios.

Abstract

Invertible neural networks (INNs) represent an important class of deep neural network architectures that have been widely used in several applications. The universal approximation properties of INNs have also been established recently. However, the approximation rate of INNs is largely missing. In this work, we provide an analysis of the capacity of a class of coupling-based INNs to approximate bi-Lipschitz continuous mappings on a compact domain, and the result shows that it can well approximate both forward and inverse maps simultaneously. Furthermore, we develop an approach for approximating bi-Lipschitz maps on infinite-dimensional spaces that simultaneously approximate the forward and inverse maps, by combining model reduction with principal component analysis and INNs for approximating the reduced map, and we analyze the overall approximation error of the approach. Preliminary numerical results show the feasibility of the approach for approximating the solution operator for parameterized second-order elliptic problems.