Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers

TL;DR

This paper introduces a novel floating-point representation with group-shared exponents and segmented mantissa storage, enabling efficient mixed-precision iterative algorithms with improved performance and convergence.

Contribution

It proposes a new floating-point format and a stepped mixed-precision iterative algorithm that reduce overhead and enhance efficiency compared to existing methods.

Findings

Significant performance improvements over existing formats.

Enhanced convergence residuals in iterative algorithms.

Efficient switching between precisions without multiple data copies.

Abstract

Iterative solvers are frequently used in scientific applications and engineering computations. However, the memory-bound Sparse Matrix-Vector (SpMV) kernel computation hinders the efficiency of iterative algorithms. As modern hardware increasingly supports low-precision computation, the mixed-precision optimization of iterative algorithms has garnered widespread attention. Nevertheless, existing mixed-precision methods pose challenges, including format conversion overhead, tight coupling between storage and computation representation, and the need to store multiple precision copies of data. This paper proposes a floating-point representation based on the group-shared exponent and segmented storage of the mantissa, enabling higher bit utilization of the representation vector and fast switches between different precisions without needing multiple data copies. Furthermore, a stepped mixed-precision iterative algorithm is proposed. Our experimental results demonstrate that, compared with existing floating-point formats, our approach significantly improves iterative algorithms' performance and convergence residuals.