A Multidimensional Assessment Method for Situated Visualization Understanding (MdamV)

TL;DR

This paper introduces MdamV, a comprehensive assessment method for visualization understanding that combines task performance, self-assessment, and critique across six dimensions, validated through a survey on climate data visualizations.

Contribution

The paper presents a novel multidimensional assessment framework for visualization understanding, integrating multiple factors and validated with a large, representative survey.

Findings

Approximately 25% of respondents struggled with simple data units.

20% felt unfamiliar with each chart type.

Self-assessed numeracy correlated with data reading performance.

Abstract

How audiences read, interpret, and critique data visualizations is mainly assessed through performance tests featuring tasks like value retrieval. Yet, other factors shown to shape visualization understanding, such as numeracy, graph familiarity, and aesthetic perception, remain underrepresented in existing instruments. To address this, we design and test a Multidimensional Assessment Method of Situated Visualization Understanding (MdamV). This method integrates task-based measures with self-perceived ability ratings and open-ended critique, applied directly to the visualizations being read. Grounded in learning sciences frameworks that view understanding as a multifaceted process, MdamV spans six dimensions: Comprehending, Decoding, Aestheticizing, Critiquing, Reading, and Contextualizing. Validation was supported by a survey (N=438) representative of Austria's population (ages 18-74, male/female split), using a line chart and a bar chart on climate data. Findings show, for example, that about a quarter of respondents indicate deficits in comprehending simple data units, roughly one in five people felt unfamiliar with each chart type, and self-assessed numeracy was significantly related to data reading performance (p=0.0004). Overall, the evaluation of MdamV demonstrates the value of assessing visualization understanding beyond performance, framing it as a situated process tied to particular visualizations.