Seeing Graphs Like Humans: Benchmarking Computational Measures and MLLMs for Similarity Assessment

TL;DR

This study evaluates how well computational measures and multimodal large language models align with human perception in graph similarity assessment, highlighting the potential of MLLMs like GPT-5 as effective, explainable proxies.

Contribution

It introduces a comprehensive benchmark comparing computational measures and MLLMs against human judgments of graph similarity, revealing MLLMs' superior alignment and interpretability.

Findings

Portrait divergence is the best computational measure but only moderately aligned with humans.

GPT-5 significantly outperforms traditional measures in matching human similarity judgments.

MLLMs, especially GPT-5, provide interpretable rationales and better mimic human perception.

Abstract

Comparing graphs to identify similarities is a fundamental task in visual analytics of graph data. To support this, visual analytics systems frequently employ quantitative computational measures to provide automated guidance. However, it remains unclear how well these measures align with subjective human visual perception, thereby offering recommendations that conflict with analysts' intuitive judgments, potentially leading to confusion rather than reducing cognitive load. Multimodal Large Language Models (MLLMs), capable of visually interpreting graphs and explaining their reasoning in natural language, have emerged as a potential alternative to address this challenge. This paper bridges the gap between human and machine assessment of graph similarity through three interconnected experiments using a dataset of 1,881 node-link diagrams. Experiment 1 collects relative similarity judgments and rationales from 32 human participants, revealing consensus on graph similarity while prioritizing global shapes and edge densities over exact topological details. Experiment 2 benchmarks 16 computational measures against these human judgments, identifying Portrait divergence as the best-performing metric, though with only moderate alignment. Experiment 3 evaluates the potential of three state-of-the-art MLLMs (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5) as perceptual proxies. The results demonstrate that MLLMs, particularly GPT-5, significantly outperform traditional measures in aligning with human graph similarity perception and provide interpretable rationales for their decisions, whereas Claude Sonnet 4.5 shows the best computational efficiency. Our findings suggest that MLLMs hold significant promise not only as effective, explainable proxies for human perception but also as intelligent guides that can uncover subtle nuances that might be overlooked by human analysts in visual analytics systems.