ReVISit 2: A Full Experiment Life Cycle User Study Framework

TL;DR

ReVISit 2 is a comprehensive software framework that streamlines the entire lifecycle of browser-based visualization user studies, enhancing reproducibility, ease of use, and support for complex interactions.

Contribution

It introduces a unified system supporting all phases of visualization user studies, including design, debugging, data collection, analysis, and dissemination, with technical and sociotechnical features.

Findings

Proven to support publication-quality studies

Enables replay of participant interactions

Facilitates advanced interactive visualization studies

Abstract

Online user studies of visualizations, visual encodings, and interaction techniques are ubiquitous in visualization research. Yet, designing, conducting, and analyzing studies effectively is still a major burden. Although various packages support such user studies, most solutions address only facets of the experiment life cycle, make reproducibility difficult, or do not cater to nuanced study designs or interactions. We introduce reVISit 2, a software framework that supports visualization researchers at all stages of designing and conducting browser-based user studies. ReVISit supports researchers in the design, debug & pilot, data collection, analysis, and dissemination experiment phases by providing both technical affordances (such as replay of participant interactions) and sociotechnical aids (such as a mindfully maintained community of support). It is a proven system that can be (and has been) used in publication-quality studies -- which we demonstrate through a series of experimental replications. We reflect on the design of the system via interviews and an analysis of its technical dimensions. Through this work, we seek to elevate the ease with which studies are conducted, improve the reproducibility of studies within our community, and support the construction of advanced interactive studies.