StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations

TL;DR

StarNet introduces a gradient-free method for training deep generative models by solving determined linear systems, offering scalability and precise bounds for latent codes and parameters.

Contribution

It proposes a novel gradient-free training approach for deep generative models using linear systems, enhancing scalability and theoretical bounds.

Findings

Training is gradient-free and deterministic.

Method scales well with model size.

Provides least-square bounds for latent codes and parameters.

Abstract

In this paper we present an approach for training deep generative models solely based on solving determined systems of linear equations. A network that uses this approach, called a StarNet, has the following desirable properties: 1) training requires no gradient as solution to the system of linear equations is not stochastic, 2) is highly scalable when solving the system of linear equations w.r.t the latent codes, and similarly for the parameters of the model, and 3) it gives desirable least-square bounds for the estimation of latent codes and network parameters within each layer.