Learning representations by forward-propagating errors

TL;DR

This paper introduces a fast, CPU-based learning algorithm for neural networks that uses forward propagation and dual numbers, aiming to match GPU acceleration speeds without requiring GPU hardware.

Contribution

The paper presents a novel forward-propagating learning algorithm utilizing dual numbers, providing a CPU-based alternative to traditional back-propagation with comparable speed.

Findings

Achieves training speeds comparable to GPU-based methods on CPU.

Reduces computational complexity and cost of neural network training.

Demonstrates effectiveness using dual numbers in algebraic geometry.

Abstract

Back-propagation (BP) is widely used learning algorithm for neural network optimization. However, BP requires enormous computation cost and is too slow to train in central processing unit (CPU). Therefore current neural network optimizaiton is performed in graphical processing unit (GPU) with compute unified device architecture (CUDA) programming. In this paper, we propose a light, fast learning algorithm on CPU that is fast as CUDA acceleration on GPU. This algorithm is based on forward-propagating method, using concept of dual number in algebraic geometry.