Coded Computing for Fault-Tolerant Parallel QR Decomposition

TL;DR

This paper introduces a fault-tolerant parallel QR decomposition algorithm using coded computing, which enhances resilience against node failures with minimal overhead and guarantees orthogonality restoration.

Contribution

It constructs a checksum-generator matrix satisfying the post-orthogonalization condition and proposes an in-node systematic MDS coding strategy with minimal checksum requirements.

Findings

The proposed code satisfies the MDS property with high probability.

The in-node systematic MDS coding achieves the lower bound on checksum number.

Experiments show negligible overhead of the fault-tolerant framework.

Abstract

QR decomposition is an essential operation for solving linear equations and obtaining least-squares solutions. In high-performance computing systems, large-scale parallel QR decomposition often faces node faults. We address this issue by proposing a fault-tolerant algorithm that incorporates `coded computing' into the parallel Gram-Schmidt method, commonly used for QR decomposition. Coded computing introduces error-correcting codes into computational processes to enhance resilience against intermediate failures. While traditional coding strategies cannot preserve the orthogonality of $Q$, recent work has proven a post-orthogonalization condition that allows low-cost restoration of the degraded orthogonality. In this paper, we construct a checksum-generator matrix for multiple-node failures that satisfies the post-orthogonalization condition and prove that our code satisfies the maximum-distance separable (MDS) property with high probability. Furthermore, we consider in-node checksum storage setting where checksums are stored in original nodes. We obtain the minimal number of checksums required to be resilient to any $f$ failures under the in-node checksum storage, and also propose an in-node systematic MDS coding strategy that achieves the lower bound. Extensive experiments validate our theories and showcase the negligible overhead of our coded computing framework for fault-tolerant QR decomposition.