Ethical conceptual replication of visualization research considering sources of methodological bias and practical significance

TL;DR

This paper advocates for a conceptual replication approach in visualization research to assess practical significance and minimize methodological biases, thereby advancing visualization meta-science.

Contribution

It introduces a focus on conceptual replication in visualization to evaluate principles across contexts and reduce biases, fostering a more rigorous scientific foundation.

Findings

Highlights importance of practical significance in visualization principles

Emphasizes reducing methodological biases in research

Proposes a framework for visualization meta-science

Abstract

General design principles for visualization have been relatively well-established based on a combination of cognitive and perceptual theory and empirical evaluations over the past 20 years. To determine how these principles hold up across use contexts and end-users, I argue that we should emphasize conceptual replication focused on determining practical significance and reducing methodological biases. This shift in thinking aims to determine how design principles interact with methodological approaches, laying the groundwork for visualization meta-science.