Demystifying Deep Learning: A Geometric Approach to Iterative Projections

TL;DR

This paper introduces a geometric regularization framework as an alternative to traditional deep learning training, providing insights into ResNet architectures and demonstrating the ease of training complex structures.

Contribution

It proposes a semi-parametric geometric approach to deep learning that eliminates feedback loops, offering a new perspective for analyzing and training deep neural networks.

Findings

ResNet architectures relate closely to the proposed geometric approach.

The method can be trained efficiently to learn complex structures.

Preliminary results support the effectiveness of the approach.

Abstract

Parametric approaches to Learning, such as deep learning (DL), are highly popular in nonlinear regression, in spite of their extremely difficult training with their increasing complexity (e.g. number of layers in DL). In this paper, we present an alternative semi-parametric framework which foregoes the ordinarily required feedback, by introducing the novel idea of geometric regularization. We show that certain deep learning techniques such as residual network (ResNet) architecture are closely related to our approach. Hence, our technique can be used to analyze these types of deep learning. Moreover, we present preliminary results which confirm that our approach can be easily trained to obtain complex structures.