In Transformer We Trust? A Perspective on Transformer Architecture Failure Modes

TL;DR

This paper critically assesses the trustworthiness of transformer architectures across various high-stakes domains, highlighting vulnerabilities, risks, and open challenges to ensure their reliable deployment.

Contribution

It provides a comprehensive review of transformer reliability issues, including interpretability, robustness, fairness, and privacy, across multiple scientific and engineering fields.

Findings

Identifies structural vulnerabilities in transformer models.

Highlights domain-specific risks in safety-critical applications.

Outlines open research challenges for trustworthy deployment.

Abstract

Transformer architectures have revolutionized machine learning across a wide range of domains, from natural language processing to scientific computing. However, their growing deployment in high-stakes applications, such as computer vision, natural language processing, healthcare, autonomous systems, and critical areas of scientific computing including climate modeling, materials discovery, drug discovery, nuclear science, and robotics, necessitates a deeper and more rigorous understanding of their trustworthiness. In this work, we critically examine the foundational question: \textitHow trustworthy are transformer models?} We evaluate their reliability through a comprehensive review of interpretability, explainability, robustness against adversarial attacks, fairness, and privacy. We systematically examine the trustworthiness of transformer-based models in safety-critical applications spanning natural language processing, computer vision, and science and engineering domains, including robotics, medicine, earth sciences, materials science, fluid dynamics, nuclear science, and automated theorem proving; highlighting high-impact areas where these architectures are central and analyzing the risks associated with their deployment. By synthesizing insights across these diverse areas, we identify recurring structural vulnerabilities, domain-specific risks, and open research challenges that limit the reliable deployment of transformers.