Alternating Minimization for Regression with Tropical Rational Functions

TL;DR

This paper introduces an alternating minimization heuristic for regression using tropical rational functions, with applications to ReLU neural networks, demonstrating reasonable data approximation through experiments.

Contribution

It presents a novel heuristic method for regression with tropical rational functions, linking tropical algebra to neural network approximation.

Findings

Heuristic provides good approximation of input data.

Method alternates between fitting numerator and denominator.

Experimental results validate the approach.

Abstract

We propose an alternating minimization heuristic for regression over the space of tropical rational functions with fixed exponents. The method alternates between fitting the numerator and denominator terms via tropical polynomial regression, which is known to admit a closed form solution. We demonstrate the behavior of the alternating minimization method experimentally. Experiments demonstrate that the heuristic provides a reasonable approximation of the input data. Our work is motivated by applications to ReLU neural networks, a popular class of network architectures in the machine learning community which are closely related to tropical rational functions.