Heterogeneous Multilayer Generalized Operational Perceptron

TL;DR

This paper introduces a novel algorithm for learning compact, fully heterogeneous multilayer neural networks with diverse neuron types, improving classification performance over existing methods.

Contribution

It presents an efficient progressive algorithm that constructs heterogeneous networks with distinct neuron characteristics at each layer, enhancing flexibility and performance.

Findings

Outperforms related learning methods in classification tasks

Creates more compact and diverse neural network topologies

Demonstrates improved accuracy on multiple datasets

Abstract

The traditional Multilayer Perceptron (MLP) using McCulloch-Pitts neuron model is inherently limited to a set of neuronal activities, i.e., linear weighted sum followed by nonlinear thresholding step. Previously, Generalized Operational Perceptron (GOP) was proposed to extend conventional perceptron model by defining a diverse set of neuronal activities to imitate a generalized model of biological neurons. Together with GOP, Progressive Operational Perceptron (POP) algorithm was proposed to optimize a pre-defined template of multiple homogeneous layers in a layerwise manner. In this paper, we propose an efficient algorithm to learn a compact, fully heterogeneous multilayer network that allows each individual neuron, regardless of the layer, to have distinct characteristics. Based on the complexity of the problem, the proposed algorithm operates in a progressive manner on a neuronal level, searching for a compact topology, not only in terms of depth but also width, i.e., the number of neurons in each layer. The proposed algorithm is shown to outperform other related learning methods in extensive experiments on several classification problems.