Trace Ratio vs Ratio Trace Methods for Multidimensional Dimensionality Reduction

TL;DR

This paper introduces a higher-order tensor-based dimensionality reduction framework using Trace Ratio optimization, providing theoretical insights, a novel Newton-type algorithm, and demonstrating improved performance over existing methods.

Contribution

It establishes theoretical conditions for Trace Ratio solutions, connects it with Ratio Trace, and develops a tensor-based algorithm extending classical LDA.

Findings

The proposed method is efficient and robust.

It outperforms existing matrix- and tensor-based techniques.

Numerical experiments confirm the effectiveness of the approach.

Abstract

We propose a higher-order dimensionality reduction framework based on the Trace Ratio (TR) optimization problem. We establish conditions for existence and uniqueness of solutions and clarify the theoretical connection between the Trace Ratio and its surrogate, the Ratio Trace (RT) formulation. Building on these foundations, we design a Newton-type iterative algorithm that operates directly in the tensor domain via the Einstein product, avoiding data flattening and preserving multi-dimensional structure. This approach extends classical Linear Discriminant Analysis (LDA) to higher-order tensors, offering a natural generalization of trace-based dimensionality reduction from matrices to tensors. Numerical experiments on several benchmark datasets confirm the efficiency and robustness of the proposed methods, showing consistent improvements over existing matrix- and tensor-based techniques.