Complex Cognition: A New Theoretical Foundation for the Design and Evaluation of Visual Analytics Systems

TL;DR

This paper proposes adopting complex cognition theories to improve the design and evaluation of visual analytics systems, addressing the mismatch between simple cognitive models and complex analytical tasks.

Contribution

It introduces a new theoretical foundation based on complex cognition theories to enhance research validity in visual analytics system design and evaluation.

Findings

Current methods limit validity due to simple cognitive assumptions

Complex cognition theories better align with analytical tasks

Guides for future research in visual analytics design

Abstract

Current research on visual analytics systems largely follows the research paradigm of interactive system design in the field of Human-Computer Interaction (HCI), and includes key methodologies including design requirement development based on user needs, interactive system design, and system evaluation. However, most studies under this paradigm have a contradiction: there is a significant mismatch between the research methods developed for simple cognitive behaviors (e.g., color perception, the perception of spatial relationship among interactive artifacts) and research goals targeting for complex analytical behaviors (e.g., reasoning, problem-solving, decision-making). This mismatch may hurt the theoretical contributions of research studies, in particularly the internal validity of a designed system and the external validity of design methods. To address this challenge, this paper argues for a need to go beyond traditional HCI theoretical foundations and proposes to adopt complex cognition theories to build new theoretical foundations. Specifically, this paper analyzes how current design and evaluation methods in research on visual analytics systems constrain the internal and external validity of research, discusses the connections between complex cognition theories and visual analytics tasks, and explores how problem-solving theories from complex cognition can guide research on visual analytics systems.