High-Order Multilinear Discriminant Analysis via Order-$\textit{n}$ Tensor Eigendecomposition

TL;DR

This paper introduces a novel high-order multilinear discriminant analysis method based on tensor eigendecomposition, improving classification accuracy for high-dimensional tensor data in machine learning applications.

Contribution

It proposes a new tensor decomposition-based discriminant analysis framework called HOMLDA and an improved robust version RHOMLDA, addressing singularity issues in traditional methods.

Findings

HOMLDA outperforms existing Tucker-based methods in classification accuracy.

RHOMLDA effectively handles near-singular within-class scatter tensors.

Experimental results demonstrate significant improvements across multiple datasets.

Abstract

Higher-order data with high dimensionality is of immense importance in many areas of machine learning, computer vision, and video analytics. Multidimensional arrays (commonly referred to as tensors) are used for arranging higher-order data structures while keeping the natural representation of the data samples. In the past decade, great efforts have been made to extend the classic linear discriminant analysis for higher-order data classification generally referred to as multilinear discriminant analysis (MDA). Most of the existing approaches are based on the Tucker decomposition and $\textit{n}$-mode tensor-matrix products. The current paper presents a new approach to tensor-based multilinear discriminant analysis referred to as High-Order Multilinear Discriminant Analysis (HOMLDA). This approach is based upon the tensor decomposition where an order-$\textit{n}$ tensor can be written as a product of order-$\textit{n}$ tensors and has a natural extension to traditional linear discriminant analysis (LDA). Furthermore, the resulting framework, HOMLDA, might produce a within-class scatter tensor that is close to singular. Thus, computing the inverse inaccurately may distort the discriminant analysis. To address this problem, an improved method referred to as Robust High-Order Multilinear Discriminant Analysis (RHOMLDA) is introduced. Experimental results on multiple data sets illustrate that our proposed approach provides improved classification performance with respect to the current Tucker decomposition-based supervised learning methods.