Training Deep Morphological Neural Networks as Universal Approximators

TL;DR

This paper explores deep morphological neural networks, demonstrating their trainability under constraints, and introduces hybrid architectures that enhance convergence and pruning capabilities, advancing the understanding of DMNNs.

Contribution

It is the first to successfully train constrained DMNNs, proposing architectures that preserve sparsity and improve training efficiency and pruning.

Findings

Constrained DMNNs can be effectively trained.

Hybrid architectures accelerate convergence.

Morphological layers improve pruning and training speed.

Abstract

We investigate deep morphological neural networks (DMNNs). We demonstrate that despite their inherent non-linearity, "linear" activations are essential for DMNNs. To preserve their inherent sparsity, we propose architectures that constraint the parameters of the "linear" activations: For the first (resp. second) architecture, we work under the constraint that the majority of parameters (resp. learnable parameters) should be part of morphological operations. We improve the generalization ability of our networks via residual connections and weight dropout. Our proposed networks can be successfully trained, and are more prunable than linear networks. To the best of our knowledge, we are the first to successfully train DMNNs under such constraints. Finally, we propose a hybrid network architecture combining linear and morphological layers, showing empirically that the inclusion of morphological layers significantly accelerates the convergence of gradient descent with large batches.