# Implicit Regularization in Matrix Factorization

## TL;DR

This paper investigates how gradient descent on matrix factorizations implicitly promotes low-rank solutions, converging to the minimum nuclear norm solution under certain conditions.

## Contribution

It provides empirical and theoretical evidence that small step sizes and initializations near zero lead gradient descent to the minimum nuclear norm solution in matrix factorization.

## Key findings

- Gradient descent converges to minimum nuclear norm solution under specific conditions.
- Small step sizes and initializations close to zero are crucial for implicit regularization.
- Theoretical support for empirical observations on matrix factorization behavior.

## Abstract

We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix $X$ with gradient descent on a factorization of $X$. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution.