MorphoActivation: Generalizing ReLU activation function by mathematical morphology

TL;DR

This paper introduces MorphoActivation, a novel family of activation functions based on mathematical morphology, enhancing the understanding and performance of nonlinear activations and max-pooling in deep neural networks.

Contribution

It proposes a general morphological framework for activation functions and demonstrates its effectiveness on standard supervised learning benchmarks.

Findings

Improved accuracy on classical benchmarks

Validates the morphological approach for activation functions

Shows the potential of mathematical morphology in neural network design

Abstract

This paper analyses both nonlinear activation functions and spatial max-pooling for Deep Convolutional Neural Networks (DCNNs) by means of the algebraic basis of mathematical morphology. Additionally, a general family of activation functions is proposed by considering both max-pooling and nonlinear operators in the context of morphological representations. Experimental section validates the goodness of our approach on classical benchmarks for supervised learning by DCNN.