A SOM-based Gradient-Free Deep Learning Method with Convergence Analysis

TL;DR

This paper introduces a gradient-free deep learning model based on Self-Organizing Maps, incorporating residuals and convergence analysis, offering an alternative to gradient-based methods with theoretical guarantees.

Contribution

It proposes a novel deep Valued Self-Organizing Map network with convergence analysis, addressing issues of gradient descent in deep learning.

Findings

Convergence conditions derived for the proposed network.

The model integrates residuals into Self-Organizing Maps.

Theoretical bounds relate parameters to input dimension and loss.

Abstract

As gradient descent method in deep learning causes a series of questions, this paper proposes a novel gradient-free deep learning structure. By adding a new module into traditional Self-Organizing Map and introducing residual into the map, a Deep Valued Self-Organizing Map network is constructed. And analysis about the convergence performance of such a deep Valued Self-Organizing Map network is proved in this paper, which gives an inequality about the designed parameters with the dimension of inputs and the loss of prediction.