A New High Performance and Scalable SVD algorithm on Distributed Memory Systems

TL;DR

This paper presents a scalable, high-performance implementation of the Zolo-SVD algorithm on distributed memory systems, outperforming existing methods in speed and scalability, demonstrated through extensive experiments on supercomputers.

Contribution

The paper introduces a portable, scalable Zolo-SVD implementation based on Zolo-PD, offering improved speed and parallelization over existing PD algorithms like QDWH-PD.

Findings

Zolo-SVD is about twice as fast as ScaLAPACK PDGESVD.

Zolo-PD outperforms QDWH-PD by 20% on many processes.

Implementation is portable and tested on Tianhe-2 supercomputer.

Abstract

This paper introduces a high performance implementation of \texttt{Zolo-SVD} algorithm on distributed memory systems, which is based on the polar decomposition (PD) algorithm via the Zolotarev's function (\texttt{Zolo-PD}), originally proposed by Nakatsukasa and Freund [SIAM Review, 2016]. Our implementation highly relies on the routines of ScaLAPACK and therefore it is portable. Compared with the other PD algorithms such as the QR-based dynamically weighted Halley method (\texttt{QDWH-PD}), \texttt{Zolo-PD} is naturally parallelizable and has better scalability though performs more floating-point operations. When using many processes, \texttt{Zolo-PD} is usually 1.20 times faster than \texttt{QDWH-PD} algorithm, and \texttt{Zolo-SVD} can be about two times faster than the ScaLAPACK routine \texttt{\texttt{PDGESVD}}. These numerical experiments are performed on Tianhe-2 supercomputer, one of the fastest supercomputers in the world, and the tested matrices include some sparse matrices from particular applications and some randomly generated dense matrices with different dimensions. Our \texttt{QDWH-SVD} and \texttt{Zolo-SVD} implementations are freely available at https://github.com/shengguolsg/Zolo-SVD.