ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

TL;DR

This paper introduces ChartAttack, a framework to evaluate and improve the robustness of multimodal large language models against maliciously manipulated chart prompts that can mislead data interpretation.

Contribution

We propose ChartAttack and AttackViz, novel tools for testing and enhancing MLLMs' resistance to misleading chart generation and interpretation.

Findings

ChartAttack reduces MLLM QA accuracy by 17.2 points in-domain.

AttackViz effectively identifies misleading chart prompts.

Fine-tuning with AttackViz improves model robustness.

Abstract

Multimodal large language models (MLLMs) are increasingly used to automate chart generation from data tables, enabling efficient data analysis and reporting but also introducing new misuse risks. In this work, we introduce ChartAttack, a novel framework for evaluating how MLLMs can be misused to generate misleading charts at scale. ChartAttack injects misleaders into chart designs, aiming to induce incorrect interpretations of the underlying data. Furthermore, we create AttackViz, a chart question-answering (QA) dataset where each (chart specification, QA) pair is labeled with effective misleaders and their induced incorrect answers. ChartAttack significantly degrades QA performance, reducing MLLM accuracy by 17.2 points in-domain and 11.9 cross-domain. Preliminary human results (limited sample size) indicate a 20.2-point accuracy drop. Finally, we demonstrate that AttackViz can be used to fine-tune MLLMs to improve robustness against misleading charts. Our findings highlight an urgent need for robustness and security considerations in the design, evaluation, and deployment of MLLM-based chart generation systems. We make our code and data publicly available.