A spectral PALM algorithm for matrix and tensor-train based Dictionary Learning

TL;DR

This paper introduces a spectral PALM algorithm for matrix and tensor-train based Dictionary Learning, improving efficiency and convergence in image classification tasks involving complex, multidimensional data.

Contribution

It develops a novel PALM-based algorithm with second order information for matrix and tensor-train Dictionary Learning, including a new tensor formulation and convergence analysis.

Findings

Enhanced algorithm efficiency and convergence

Successful application to benchmark image datasets

Effective preservation of multidimensional data features

Abstract

Dictionary Learning (DL) is one of the leading sparsity promoting techniques in the context of image classification, where the "dictionary" matrix D of images and the sparse matrix X are determined so as to represent a redundant image dataset. The resulting constrained optimization problem is nonconvex and non-smooth, providing several computational challenges for its solution. To preserve multidimensional data features, various tensor DL formulations have been introduced, adding to the problem complexity. We develop a new alternating algorithm for the solution of the DL problem both in the matrix and tensor frameworks; in the latter case a new formulation based on Tensor-Train decompositions is also proposed. The new method belongs to the Proximal Alternating Linearized Minimization (PALM) algorithmic family, with the inclusion of second order information to enhance efficiency. We discuss a rigorous convergence analysis, and report on the new method performance on the image classification of several benchmark datasets.