Methods for the Design and Evaluation of HCI+NLP Systems

TL;DR

This paper proposes five interdisciplinary methods to improve the design and evaluation of HCI and NLP systems, aiming to foster collaboration and shared progress between the two fields.

Contribution

It introduces five novel methodologies at the intersection of HCI and NLP, integrating evaluation approaches from both disciplines.

Findings

Five new methodological proposals for HCI+NLP systems

Enhanced interdisciplinary collaboration strategies

Frameworks for integrating evaluation methods

Abstract

HCI and NLP traditionally focus on different evaluation methods. While HCI involves a small number of people directly and deeply, NLP traditionally relies on standardized benchmark evaluations that involve a larger number of people indirectly. We present five methodological proposals at the intersection of HCI and NLP and situate them in the context of ML-based NLP models. Our goal is to foster interdisciplinary collaboration and progress in both fields by emphasizing what the fields can learn from each other.