Parameter efficient dendritic-tree neurons outperform perceptrons

TL;DR

This paper introduces biologically inspired dendritic-tree neurons that enhance perceptron performance by leveraging complex input branching and dropout, leading to more accurate and generalizable models in image classification tasks.

Contribution

It proposes a novel dendritic neuron model with input branching and dropout, and provides a PyTorch implementation for improved neural network architectures.

Findings

Dendritic neurons outperform perceptrons on MNIST classification.

Parameter-efficient non-linear input architectures are effective.

The approach improves accuracy and generalization in neural networks.

Abstract

Biological neurons are more powerful than artificial perceptrons, in part due to complex dendritic input computations. Inspired to empower the perceptron with biologically inspired features, we explore the effect of adding and tuning input branching factors along with input dropout. This allows for parameter efficient non-linear input architectures to be discovered and benchmarked. Furthermore, we present a PyTorch module to replace multi-layer perceptron layers in existing architectures. Our initial experiments on MNIST classification demonstrate the accuracy and generalization improvement of dendritic neurons compared to existing perceptron architectures.