Metabolize Neural Network

TL;DR

This paper introduces MetaNet, a neural network model inspired by cellular metabolism, which self-grows and prunes neurons during training to optimize resource use and improve learning efficiency.

Contribution

It proposes a novel neuron proliferation and autophagy mechanism for self-constructing neural networks, reducing resource waste and automating neuron count determination.

Findings

MetaNet effectively adapts neuron numbers during training.

The method improves resource efficiency in neural network training.

Performance verified on MNIST, Fashion-MNIST, and CIFAR-10 datasets.

Abstract

The metabolism of cells is the most basic and important part of human function. Neural networks in deep learning stem from neuronal activity. It is self-evident that the significance of metabolize neuronal network(MetaNet) in model construction. In this study, we explore neuronal metabolism for shallow network from proliferation and autophagy two aspects. First, we propose different neuron proliferate methods that constructive the selfgrowing network in metabolism cycle. Proliferate neurons alleviate resources wasting and insufficient model learning problem when network initializes more or less parameters. Then combined with autophagy mechanism in the process of model self construction to ablate under-expressed neurons. The MetaNet can automatically determine the number of neurons during training, further, save more resource consumption. We verify the performance of the proposed methods on datasets: MNIST, Fashion-MNIST and CIFAR-10.