Vulnerability of LLMs to Vertically Aligned Text Manipulations

TL;DR

This paper investigates how vertically aligned text manipulations significantly impair the performance of large language models in classification tasks, revealing vulnerabilities related to tokenization and attention mechanisms.

Contribution

It provides a comprehensive analysis of the impact of vertical text input on various LLMs and explores underlying causes, offering insights into model vulnerabilities and potential mitigation strategies.

Findings

Vertical text significantly reduces LLM accuracy in classification.

Chain-of-Thought reasoning does not mitigate vertical input issues.

Tokenization and attention mechanisms are key causes of vulnerability.

Abstract

Vertical text input is commonly encountered in various real-world applications, such as mathematical computations and word-based Sudoku puzzles. While current large language models (LLMs) have excelled in natural language tasks, they remain vulnerable to variations in text formatting. Recent research demonstrates that modifying input formats, such as vertically aligning words for encoder-based models, can substantially lower accuracy in text classification tasks. While easily understood by humans, these inputs can significantly mislead models, posing a potential risk of bypassing detection in real-world scenarios involving harmful or sensitive information. With the expanding application of LLMs, a crucial question arises: Do decoder-based LLMs exhibit similar vulnerabilities to vertically formatted text input? In this paper, we investigate the impact of vertical text input on the performance of various LLMs across multiple text classification datasets and analyze the underlying causes. Our findings are as follows: (i) Vertical text input significantly degrades the accuracy of LLMs in text classification tasks. (ii) Chain-of-Thought (CoT) reasoning does not help LLMs recognize vertical input or mitigate its vulnerability, but few-shot learning with careful analysis does. (iii) We explore the underlying cause of the vulnerability by analyzing the inherent issues in tokenization and attention matrices.