In Defence of Visual Analytics Systems: Replies to Critics

TL;DR

This paper defends the value of visual analytics systems by presenting interview studies that gather and respond to criticisms, aiming to enhance research rigor and foster constructive discussion in the field.

Contribution

It introduces two interview studies that identify criticisms and responses regarding visual analytics research, providing a foundation for improving scientific rigor and community dialogue.

Findings

Identified 36 common criticisms of VA systems.

Collected researcher responses to criticisms.

Highlighted the need for rigorous and inclusive VA research.

Abstract

The last decade has witnessed many visual analytics (VA) systems that make successful applications to wide-ranging domains like urban analytics and explainable AI. However, their research rigor and contributions have been extensively challenged within the visualization community. We come in defence of VA systems by contributing two interview studies for gathering critics and responses to those criticisms. First, we interview 24 researchers to collect criticisms the review comments on their VA work. Through an iterative coding and refinement process, the interview feedback is summarized into a list of 36 common criticisms. Second, we interview 17 researchers to validate our list and collect their responses, thereby discussing implications for defending and improving the scientific values and rigor of VA systems. We highlight that the presented knowledge is deep, extensive, but also imperfect, provocative, and controversial, and thus recommend reading with an inclusive and critical eye. We hope our work can provide thoughts and foundations for conducting VA research and spark discussions to promote the research field forward more rigorously and vibrantly.