Min-Max-Plus Neural Networks

TL;DR

This paper introduces Min-Max-Plus Neural Networks (MMP-NNs), a novel model based on tropical arithmetic operations, which enhances nonlinearity and universality in function approximation with efficient training algorithms.

Contribution

The paper proposes MMP-NNs with layered min-plus and max-plus operations, demonstrating their universality and developing training and normalization algorithms.

Findings

MMP-NNs are universal approximators of continuous functions.

They can operate with minimal or no multiplication operations.

The proposed training algorithms improve convergence rates.

Abstract

We present a new model of neural networks called Min-Max-Plus Neural Networks (MMP-NNs) based on operations in tropical arithmetic. In general, an MMP-NN is composed of three types of alternately stacked layers, namely linear layers, min-plus layers and max-plus layers. Specifically, the latter two types of layers constitute the nonlinear part of the network which is trainable and more sophisticated compared to the nonlinear part of conventional neural networks. In addition, we show that with higher capability of nonlinearity expression, MMP-NNs are universal approximators of continuous functions, even when the number of multiplication operations is tremendously reduced (possibly to none in certain extreme cases). Furthermore, we formulate the backpropagation algorithm in the training process of MMP-NNs and introduce an algorithm of normalization to improve the rate of convergence in training.