Insights From Experiments With Rigor in an EvoBio Design Study

TL;DR

This paper reports on a year-long design study with evolutionary biologists, introducing new visualization techniques, methodological insights, and reporting devices to enhance rigor and knowledge contribution in visualization research.

Contribution

It presents a new interpretivist perspective on design study, with six criteria for rigor, and demonstrates their application through developing visualization techniques and methodological recommendations.

Findings

Two novel visualization techniques for phylogenetic data

Three methodological recommendations for design studies

Two reporting devices for interpretivist design studies

Abstract

Design study is an established approach of conducting problem-driven visualization research. The academic visualizationcommunity has produced a large body of work for reporting on design studies, informed by a handful of theoretical frameworks, andapplied to a broad range of application areas. The result is an abundance of reported insights into visualization design, with anemphasis on novel visualization techniques and systems as the primary contribution of these studies. In recent work we proposeda new, interpretivist perspective on design study and six companion criteria for rigor that highlight the opportunities for researchersto contribute knowledge that extends beyond visualization idioms and software. In this work we conducted a year-long collaborationwith evolutionary biologists to develop an interactive tool for visual exploration of multivariate datasets and phylogenetic trees. Duringthis design study we experimented with methods to support three of the rigor criteria:ABUNDANT,REFLEXIVE, andTRANSPARENT. As aresult we contribute two novel visualization techniques for the analysis of multivariate phylogenetic datasets, three methodologicalrecommendations for conducting design studies drawn from reflections over our process of experimentation, and two writing devices forreporting interpretivist design study. We offer this work as an example for implementing the rigor criteria to produce a diverse range ofknowledge contributions.