Convex Optimization For Non-Convex Problems via Column Generation

TL;DR

This paper introduces a novel convex optimization approach using column generation to approximate complex structured objects with primitive components, effectively handling large-scale tensor approximations.

Contribution

It proposes a new method combining column generation and convex optimization for structured object approximation, including low-rank tensor approximation, with efficient inference in the dual.

Findings

Effective low-rank tensor approximations achieved

Utilizes L1 regularization for sparse primitive usage

Demonstrates scalability to large 3-way tensors

Abstract

We apply column generation to approximating complex structured objects via a set of primitive structured objects under either the cross entropy or L2 loss. We use L1 regularization to encourage the use of few structured primitive objects. We attack approximation using convex optimization over an infinite number of variables each corresponding to a primitive structured object that are generated on demand by easy inference in the Lagrangian dual. We apply our approach to producing low rank approximations to large 3-way tensors.