H is for Human and How (Not) To Evaluate Qualitative Research in HCI

TL;DR

This paper discusses the challenges of evaluating qualitative research in HCI, emphasizing interpretivist approaches over positivistic metrics, and proposes criteria for quality assurance without relying on quantification.

Contribution

It clarifies the philosophical differences in research paradigms and offers practical criteria for assessing qualitative studies in HCI.

Findings

Highlights the contrast between positivism and interpretivism in research evaluation

Proposes five criteria for qualitative research quality assurance

Encourages evaluation methods aligned with interpretivist principles

Abstract

Concern has recently been expressed by HCI researchers as to the inappropriate treatment of qualitative studies through a positivistic mode of evaluation that places emphasis on metrics and measurement. This contrasts with the nature of qualitative research, which privileges interpretation and understanding over quantification. This paper explains the difference between positivism and interpretivism, the limits of quantification in human science, the distinctive contribution of qualitative research, and how quality assurance might be provided for in the absence of numbers via five basic criteria that reviewers may use to evaluate qualitative studies on their own terms.