The Power of Order: Fooling LLMs with Adversarial Table Permutations

TL;DR

This paper uncovers a significant vulnerability in large language models where simple, semantically-invariant permutations of tabular data can cause incorrect outputs, highlighting a need for more robust models.

Contribution

It introduces Adversarial Table Permutation, a novel gradient-based attack method to systematically identify permutations that disrupt LLM performance on tabular data.

Findings

Permutations can significantly degrade LLM performance.

The attack is effective across various model sizes and architectures.

Current models lack robustness to table structure variations.

Abstract

Large Language Models have achieved remarkable success and are increasingly deployed in critical applications involving tabular data, such as Table Question Answering. However, their robustness to the structure of this input remains a critical, unaddressed question. This paper demonstrates that modern LLMs exhibit a significant vulnerability to the layout of tabular data. Specifically, we show that semantically-invariant permutations of rows and columns - rearrangements that do not alter the table's underlying information - are sometimes sufficient to cause incorrect or inconsistent model outputs. To systematically probe this vulnerability, we introduce Adversarial Table Permutation, a novel, gradient-based attack that efficiently identifies worst-case permutations designed to maximally disrupt model performance. Our extensive experiments demonstrate that ATP significantly degrades the performance of a wide range of LLMs. This reveals a pervasive vulnerability across different model sizes and architectures, including the most recent and popular models. Our findings expose a fundamental weakness in how current LLMs process structured data, underscoring the urgent need to develop permutation-robust models for reliable, real-world applications.