Parameter Sensitivity Analysis of the SparTen High Performance Sparse Tensor Decomposition Software: Extended Analysis

TL;DR

This paper investigates how parameter choices affect the convergence of the SparTen tensor decomposition library across various datasets and hardware, providing insights into its robustness and optimal settings.

Contribution

It offers a comprehensive sensitivity analysis of SparTen's parameters, extending previous work by testing across multiple datasets, architectures, and initializations.

Findings

Default parameters may not be optimal for all data types.

Algorithm convergence is sensitive to parameter variations.

Robust profiles of convergence behavior across datasets and hardware.

Abstract

Tensor decomposition models play an increasingly important role in modern data science applications. One problem of particular interest is fitting a low-rank Canonical Polyadic (CP) tensor decomposition model when the tensor has sparse structure and the tensor elements are nonnegative count data. SparTen is a high-performance C++ library which computes a low-rank decomposition using different solvers: a first-order quasi-Newton or a second-order damped Newton method, along with the appropriate choice of runtime parameters. Since default parameters in SparTen are tuned to experimental results in prior published work on a single real-world dataset conducted using MATLAB implementations of these methods, it remains unclear if the parameter defaults in SparTen are appropriate for general tensor data. Furthermore, it is unknown how sensitive algorithm convergence is to changes in the input parameter values. This report addresses these unresolved issues with large-scale experimentation on three benchmark tensor data sets. Experiments were conducted on several different CPU architectures and replicated with many initial states to establish generalized profiles of algorithm convergence behavior.