# DANTE: Deep AlterNations for Training nEural networks

## TL;DR

DANTE introduces an alternative neural network training method based on alternating minimization and bi-quasi-convex optimization, offering a promising and competitive approach compared to traditional backpropagation.

## Contribution

It presents a novel training framework for neural networks using alternating minimization and bi-quasi-convexity, applicable to various activation functions and network depths.

## Key findings

- Competitive training quality compared to backpropagation
- Effective for both differentiable and non-differentiable activations
- Potentially faster training speeds

## Abstract

We present DANTE, a novel method for training neural networks using the alternating minimization principle. DANTE provides an alternate perspective to traditional gradient-based backpropagation techniques commonly used to train deep networks. It utilizes an adaptation of quasi-convexity to cast training a neural network as a bi-quasi-convex optimization problem. We show that for neural network configurations with both differentiable (e.g. sigmoid) and non-differentiable (e.g. ReLU) activation functions, we can perform the alternations effectively in this formulation. DANTE can also be extended to networks with multiple hidden layers. In experiments on standard datasets, neural networks trained using the proposed method were found to be promising and competitive to traditional backpropagation techniques, both in terms of quality of the solution, as well as training speed.