# Singular Value Decomposition and Neural Networks

## TL;DR

This paper explores the connection between Singular Value Decomposition (SVD) and neural networks, highlighting SVD's role as a linear analogy and an effective initialization method for training neural networks.

## Contribution

It introduces the conceptual link between SVD and neural networks and demonstrates its utility in improving network training initialization.

## Key findings

- SVD serves as a linear analogy to neural networks.
- Using SVD as an initial guess improves optimization results.
- SVD-based initialization leads to better training performance.

## Abstract

Singular Value Decomposition (SVD) constitutes a bridge between the linear algebra concepts and multi-layer neural networks---it is their linear analogy. Besides of this insight, it can be used as a good initial guess for the network parameters, leading to substantially better optimization results.