Max and Coincidence Neurons in Neural Networks

TL;DR

This paper explores the integration of max and coincidence neurons into neural networks, optimizing their architecture through neural architecture search to improve efficiency and accuracy.

Contribution

It introduces a novel approach to include diverse neuron types in network design and demonstrates their benefits through optimized architectures.

Findings

2% average accuracy improvement

25% reduction in network size

Development of a signal-processing ResNet

Abstract

Network design has been a central topic in machine learning. Large amounts of effort have been devoted towards creating efficient architectures through manual exploration as well as automated neural architecture search. However, todays architectures have yet to consider the diversity of neurons and the existence of neurons with specific processing functions. In this work, we optimize networks containing models of the max and coincidence neurons using neural architecture search, and analyze the structure, operations, and neurons of optimized networks to develop a signal-processing ResNet. The developed network achieves an average of 2% improvement in accuracy and a 25% improvement in network size across a variety of datasets, demonstrating the importance of neuronal functions in creating compact, efficient networks.