High-Performance Star-M SVD for Big Data Compression

TL;DR

This paper introduces a high-performance, parallel implementation of star-M SVD for tensor-based data compression, enabling efficient handling of large scientific datasets with optimal accuracy.

Contribution

It presents a novel shared-memory parallel software solution for star-M SVD, improving efficiency and scalability over existing implementations.

Findings

Achieved efficient parallel performance on large datasets.

Demonstrated superior compression with minimal accuracy loss.

Enabled practical application of star-M SVD in big data contexts.

Abstract

In the era of big data, effectively compressing large datasets while performing complex mathematical operations is crucial. Tensor-based decomposition methods have shown superior compression capabilities with minimal loss of accuracy compared to traditional matrix methods. Under the star-M tensor framework, tensors can be decomposed in a matrix-mimetic way, including using the star-M SVD. This tensor SVD has optimality guarantees and has shown exceptional performance on specific types of data, but software implementations have been mostly limited to productivity-oriented languages. In this work, we present our development of a shared-memory parallel, high-performance solution designed to efficiently implement the underlying algorithms. This software will enable optimal compression of extensive scientific datasets, paving the way for enhanced data analysis and insights.