Difference of Convex (DC) approach for neural network approximation with uniform loss function

TL;DR

This paper explores the use of a difference convex (DC) programming approach for neural network approximation, demonstrating its efficiency in uniform (minimax) approximation tasks compared to traditional methods.

Contribution

It introduces the application of DC programming to uniform neural network approximation, showing its effectiveness as an alternative to existing optimization techniques.

Findings

DC programming is efficient for minimax neural network approximation.

Compared to ADAMAX, DC approach shows competitive performance.

Numerical experiments validate the effectiveness of the DC method.

Abstract

Neural networks (NNs) can be viewed as approximation tools. Traditionally, NNs are relying on gradient and stochastic gradient (SG) methods. There are a number of available computational packages for constructing least squares approximations, while uniform (minimax) approximations are hard due to their nonsmooth nature. It was recently demonstrated that a difference convex (DC) programming approach is an efficient alternative optimiser for NNs. In this paper, we demonstrate that a DC programming approach is also efficient for minimax approximation. In our numerical experiments, we compare a DC-programming approach and ADAMAX, a commonly used method for minimax NN approximations.