ALBERTA: ALgorithm-Based Error Resilience in Transformer Architectures

TL;DR

ALBERTA is a framework that enhances the reliability of transformer architectures in safety-critical applications by efficiently detecting and correcting errors with minimal overhead, ensuring high error coverage.

Contribution

It introduces a novel resilience analysis and protection method for transformers, focusing on vulnerable GEMM layers with checksum-based error detection and self-correction mechanisms.

Findings

Achieves over 99% error coverage with minimal overhead.

Effectively protects GEMM layers in transformer models.

Applicable across various GPU architectures and precisions.

Abstract

Vision Transformers are being increasingly deployed in safety-critical applications that demand high reliability. It is crucial to ensure the correctness of their execution in spite of potential errors such as transient hardware errors. We propose a novel algorithm-based resilience framework called ALBERTA that allows us to perform end-to-end resilience analysis and protection of transformer-based architectures. First, our work develops an efficient process of computing and ranking the resilience of transformers layers. We find that due to the large size of transformer models, applying traditional network redundancy to a subset of the most vulnerable layers provides high error coverage albeit with impractically high overhead. We address this shortcoming by providing a software-directed, checksum-based error detection technique aimed at protecting the most vulnerable general matrix multiply (GEMM) layers in the transformer models that use either floating-point or integer arithmetic. Results show that our approach achieves over 99% coverage for errors that result in a mismatch with less than 0.2% and 0.01% computation and memory overheads, respectively. Lastly, we present the applicability of our framework in various modern GPU architectures under different numerical precisions. We introduce an efficient self-correction mechanism for resolving erroneous detection with an average of less than 2% overhead per error.