Optimization Landscape of Tucker Decomposition

TL;DR

This paper analyzes the optimization landscape of Tucker decomposition, showing all local minima are globally optimal if the tensor has an exact decomposition, and proposes a polynomial-time local search algorithm.

Contribution

It characterizes the nonconvex optimization landscape of Tucker decomposition and introduces an efficient algorithm for finding approximate solutions.

Findings

All local minima are globally optimal for tensors with exact Tucker decompositions.

A polynomial-time local search algorithm can find approximate global optima.

The landscape analysis supports the effectiveness of local search methods in practice.

Abstract

Tucker decomposition is a popular technique for many data analysis and machine learning applications. Finding a Tucker decomposition is a nonconvex optimization problem. As the scale of the problems increases, local search algorithms such as stochastic gradient descent have become popular in practice. In this paper, we characterize the optimization landscape of the Tucker decomposition problem. In particular, we show that if the tensor has an exact Tucker decomposition, for a standard nonconvex objective of Tucker decomposition, all local minima are also globally optimal. We also give a local search algorithm that can find an approximate local (and global) optimal solution in polynomial time.