Graph Drawing for LLMs: An Empirical Evaluation

TL;DR

This paper empirically evaluates how different graph drawing layouts, aesthetics, and prompting techniques influence the performance of Large Language Models in graph-related tasks involving visual inputs.

Contribution

It provides a comprehensive experimental analysis of factors affecting LLM performance on visual graph tasks, highlighting the importance of layout, readability, and prompting strategies.

Findings

Proper layout and readability significantly improve model performance.

Prompting technique selection is crucial and challenging.

Optimizing visual input enhances LLM effectiveness in graph tasks.

Abstract

Our work contributes to the fast-growing literature on the use of Large Language Models (LLMs) to perform graph-related tasks. In particular, we focus on usage scenarios that rely on the visual modality, feeding the model with a drawing of the graph under analysis. We investigate how the model's performance is affected by the chosen layout paradigm, the aesthetics of the drawing, and the prompting technique used for the queries. We formulate three corresponding research questions and present the results of a thorough experimental analysis. Our findings reveal that choosing the right layout paradigm and optimizing the readability of the input drawing from a human perspective can significantly improve the performance of the model on the given task. Moreover, selecting the most effective prompting technique is a challenging yet crucial task for achieving optimal performance.