Orientation-Aware Sparse Tensor PCA for Efficient Unsupervised Feature Selection

TL;DR

This paper introduces an orientation-aware sparse tensor PCA method that leverages tensor decomposition and a novel tensor-tensor product to improve unsupervised feature selection, especially for orientation-specific data like time series.

Contribution

It extends sparse PCA to tensor form using orientation-dependent tensor-tensor products, enabling more accurate and flexible feature selection in tensor data.

Findings

Demonstrates superior accuracy over state-of-the-art methods.

Achieves remarkable computational efficiency.

Effective on diverse real-world datasets.

Abstract

Recently, introducing Tensor Decomposition (TD) techniques into unsupervised feature selection (UFS) has been an emerging research topic. A tensor structure is beneficial for mining the relations between different modes and helps relieve the computation burden. However, while existing methods exploit TD to preserve the data tensor structure, they do not consider the influence of data orientation and thus have difficulty in handling orientation-specific data such as time series. To solve the above problem, we utilize the orientation-dependent tensor-tensor product from Tensor Singular Value Decomposition based on *M-product (T-SVDM) and extend the one-dimensional Sparse Principal Component Analysis (SPCA) to a tensor form. The proposed sparse tensor PCA model can constrain sparsity at the specified mode and yield sparse tensor principal components, enhancing flexibility and accuracy in learning feature relations. To ensure fast convergence and a flexible description of feature correlation, we develop a convex version specially designed for general UFS tasks and propose an efficient slice-by-slice algorithm that performs dual optimization in the transform domain. Experimental results on real-world datasets demonstrate the effectiveness and remarkable computational efficiency of the proposed method for tensor data of diverse structures over the state-of-the-art. When transform axes align with feature distribution patterns, our method is promising for various applications. The codes related to our proposed methods and the experiments are available at https://github.com/zjj20212035/STPCA.git.