An Alternative Practice of Tropical Convolution to Traditional Convolutional Neural Networks

TL;DR

This paper introduces Tropical Convolutional Neural Networks (TCNNs), which replace multiplications with min/max operations to reduce computational cost and potentially increase nonlinear fitting ability, showing promising results on image classification datasets.

Contribution

The paper proposes a novel CNN variant using tropical convolutions, replacing multiplications with min/max operations, and demonstrates its effectiveness on standard image datasets.

Findings

TCNN achieves higher expressive power than traditional CNNs on MNIST and CIFAR10.

TCNN shows comparable robustness to noise environments as conventional CNNs.

Tropical convolutions reduce computational complexity by replacing multiplications with min/max operations.

Abstract

Convolutional neural networks (CNNs) have been used in many machine learning fields. In practical applications, the computational cost of convolutional neural networks is often high with the deepening of the network and the growth of data volume, mostly due to a large amount of multiplication operations of floating-point numbers in convolution operations. To reduce the amount of multiplications, we propose a new type of CNNs called Tropical Convolutional Neural Networks (TCNNs) which are built on tropical convolutions in which the multiplications and additions in conventional convolutional layers are replaced by additions and min/max operations respectively. In addition, since tropical convolution operators are essentially nonlinear operators, we expect TCNNs to have higher nonlinear fitting ability than conventional CNNs. In the experiments, we test and analyze several different architectures of TCNNs for image classification tasks in comparison with similar-sized conventional CNNs. The results show that TCNN can achieve higher expressive power than ordinary convolutional layers on the MNIST and CIFAR10 image data set. In different noise environments, there are wins and losses in the robustness of TCNN and ordinary CNNs.