A Variational Inequality Model for Learning Neural Networks

TL;DR

This paper introduces a variational inequality framework for training neural networks, providing convergence guarantees and an efficient block-iterative algorithm as an alternative to traditional non-convex optimization methods.

Contribution

It proposes a novel variational inequality approach for neural network training, ensuring convergence and efficiency, which differs from standard optimization-based methods.

Findings

Algorithm guarantees convergence to true solutions.

Block-iterative structure enhances computational efficiency.

Numerical application demonstrates practical effectiveness.

Abstract

Neural networks have become ubiquitous tools for solving signal and image processing problems, and they often outperform standard approaches. Nevertheless, training neural networks is a challenging task in many applications. The prevalent training procedure consists of minimizing highly non-convex objectives based on data sets of huge dimension. In this context, current methodologies are not guaranteed to produce global solutions. We present an alternative approach which foregoes the optimization framework and adopts a variational inequality formalism. The associated algorithm guarantees convergence of the iterates to a true solution of the variational inequality and it possesses an efficient block-iterative structure. A numerical application is presented.