Theoretical and Experimental Analyses of Tensor-Based Regression and Classification

TL;DR

This paper provides a comprehensive theoretical and experimental investigation of tensor-based regression and classification, focusing on tensor norm regularization, optimization methods, and performance comparison with vector- and matrix-based approaches.

Contribution

It introduces dual optimization algorithms for tensor norms and derives excess risk bounds, advancing understanding of tensor regularization in machine learning.

Findings

Tensor-based methods outperform vector- and matrix-based methods in experiments.

Efficient dual optimization algorithms are proposed for tensor norm regularization.

Theoretical excess risk bounds clarify the behavior of different tensor norms.

Abstract

We theoretically and experimentally investigate tensor-based regression and classification. Our focus is regularization with various tensor norms, including the overlapped trace norm, the latent trace norm, and the scaled latent trace norm. We first give dual optimization methods using the alternating direction method of multipliers, which is computationally efficient when the number of training samples is moderate. We then theoretically derive an excess risk bound for each tensor norm and clarify their behavior. Finally, we perform extensive experiments using simulated and real data and demonstrate the superiority of tensor-based learning methods over vector- and matrix-based learning methods.