Growing axons: greedy learning of neural networks with application to function approximation

TL;DR

This paper introduces a greedy, incremental method for training deep neural networks by adding one basis function at a time, leading to more accurate function approximations.

Contribution

It presents a novel greedy learning approach that grows neural networks incrementally, improving approximation accuracy over traditional methods.

Findings

More accurate function approximations achieved

Effective for various model problems

Simplifies training process

Abstract

We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural network by one neuron at a time) allows us to compute much more accurate approximants for several model problems in function approximation.