A Kernelizable Primal-Dual Formulation of the Multilinear Singular Value Decomposition

TL;DR

This paper introduces a kernelizable primal-dual formulation of the Multilinear Singular Value Decomposition (MLSVD), enabling nonlinear extensions and computational advantages for high-dimensional tensor data analysis.

Contribution

It derives a primal-dual formulation of MLSVD that generalizes PCA and SVD, and proposes a nonlinear extension using kernel methods.

Findings

Provides a primal formulation for MLSVD that improves computational efficiency.

Introduces a kernelized nonlinear MLSVD extension.

Discusses applications in signal analysis and deep learning.

Abstract

The ability to express a learning task in terms of a primal and a dual optimization problem lies at the core of a plethora of machine learning methods. For example, Support Vector Machine (SVM), Least-Squares Support Vector Machine (LS-SVM), Ridge Regression (RR), Lasso Regression (LR), Principal Component Analysis (PCA), and more recently Singular Value Decomposition (SVD) have all been defined either in terms of primal weights or in terms of dual Lagrange multipliers. The primal formulation is computationally advantageous in the case of large sample size while the dual is preferred for high-dimensional data. Crucially, said learning problems can be made nonlinear through the introduction of a feature map in the primal problem, which corresponds to applying the kernel trick in the dual. In this paper we derive a primal-dual formulation of the Multilinear Singular Value Decomposition (MLSVD), which recovers as special cases both PCA and SVD. Besides enabling computational gains through the derived primal formulation, we propose a nonlinear extension of the MLSVD using feature maps, which results in a dual problem where a kernel tensor arises. We discuss potential applications in the context of signal analysis and deep learning.