Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention

TL;DR

This paper investigates how visual literacy influences attention patterns in data visualization, proposing models to predict literacy levels from attention data and vice versa, thereby enabling personalized visualization experiences.

Contribution

It introduces two models, Lit2Sal and Sal2Lit, that predict visual attention based on literacy levels and literacy from attention data, respectively, advancing personalized visualization design.

Findings

Lit2Sal outperforms existing saliency models with literacy-aware features.

Sal2Lit predicts visual literacy with 86% accuracy from attention maps.

Distinct attention patterns correlate with literacy levels.

Abstract

Accounting for individual differences can improve the effectiveness of visualization design. While the role of visual attention in visualization interpretation is well recognized, existing work often overlooks how this behavior varies based on visual literacy levels. Based on data from a 235-participant user study covering three visualization tests (mini-VLAT, CALVI, and SGL), we show that distinct attention patterns in visual data exploration can correlate with participants' literacy levels: While experts (high-scorers) generally show a strong attentional focus, novices (low-scorers) focus less and explore more. We then propose two computational models leveraging these insights: Lit2Sal -- a novel visual saliency model that predicts observer attention given their visualization literacy level, and Sal2Lit -- a model to predict visual literacy from human visual attention data. Our quantitative and qualitative evaluation demonstrates that Lit2Sal outperforms state-of-the-art saliency models with literacy-aware considerations. Sal2Lit predicts literacy with 86% accuracy using a single attention map, providing a time-efficient supplement to literacy assessment that only takes less than a minute. Taken together, our unique approach to consider individual differences in salience models and visual attention in literacy assessments paves the way for new directions in personalized visual data communication to enhance understanding.