Deep learning: Technical introduction

TL;DR

This paper provides a detailed technical overview of the three main neural network architectures—Feedforward, Convolutional, and Recurrent—including their fundamental components, forward pass, and backpropagation update rules.

Contribution

It offers a comprehensive, pedagogical explanation of core neural network architectures and their training algorithms, suitable for learners and practitioners.

Findings

Detailed derivation of forward pass for each architecture

Explicit backpropagation update rules

Clear presentation of fundamental network components

Abstract

This note presents in a technical though hopefully pedagogical way the three most common forms of neural network architectures: Feedforward, Convolutional and Recurrent. For each network, their fundamental building blocks are detailed. The forward pass and the update rules for the backpropagation algorithm are then derived in full.