A foundation for exact binarized morphological neural networks

TL;DR

This paper introduces a mathematically grounded framework for binarized morphological neural networks, enabling efficient deep learning with reduced computation and energy costs, and proposes new approximation methods and regularization techniques.

Contribution

It develops a robust theoretical framework for binarizing ConvNets using Mathematical Morphology and introduces novel approximation and regularization methods to enhance training.

Findings

Model can learn complex morphological networks

Binarized ConvNets maintain performance under certain conditions

Proposed methods improve training robustness

Abstract

Training and running deep neural networks (NNs) often demands a lot of computation and energy-intensive specialized hardware (e.g. GPU, TPU...). One way to reduce the computation and power cost is to use binary weight NNs, but these are hard to train because the sign function has a non-smooth gradient. We present a model based on Mathematical Morphology (MM), which can binarize ConvNets without losing performance under certain conditions, but these conditions may not be easy to satisfy in real-world scenarios. To solve this, we propose two new approximation methods and develop a robust theoretical framework for ConvNets binarization using MM. We propose as well regularization losses to improve the optimization. We empirically show that our model can learn a complex morphological network, and explore its performance on a classification task.