On the Equivalence between Implicit and Explicit Neural Networks: A High-dimensional Viewpoint

TL;DR

This paper explores the theoretical relationship between implicit and explicit neural networks in high-dimensional settings, showing their equivalence through kernel analysis.

Contribution

It provides a high-dimensional theoretical framework establishing the equivalence between implicit and explicit neural networks.

Findings

High-dimensional conjugate kernels derived for implicit networks

Equivalence between implicit and explicit networks established in high dimensions

Theoretical insights into neural network behavior in high-dimensional regimes

Abstract

Implicit neural networks have demonstrated remarkable success in various tasks. However, there is a lack of theoretical analysis of the connections and differences between implicit and explicit networks. In this paper, we study high-dimensional implicit neural networks and provide the high dimensional equivalents for the corresponding conjugate kernels and neural tangent kernels. Built upon this, we establish the equivalence between implicit and explicit networks in high dimensions.