Generative User-Experience Research for Developing Domain-specific Natural Language Processing Applications

TL;DR

This paper introduces a novel generative UX research methodology that involves domain users early and late in NLP application development, enhancing user trust and system relevance.

Contribution

It proposes an integrated approach combining generative UX research with NLP development, focusing on early ideation and final evaluation stages.

Findings

Involving domain experts increases trust in NLP applications.

The methodology effectively balances data-driven and user-driven development.

Case study demonstrates improved user engagement and system relevance.

Abstract

User experience (UX) is a part of human-computer interaction (HCI) research and focuses on increasing intuitiveness, transparency, simplicity, and trust for the system users. Most UX research for machine learning (ML) or natural language processing (NLP) focuses on a data-driven methodology. It engages domain users mainly for usability evaluation. Moreover, more typical UX methods tailor the systems towards user usability, unlike learning about the user needs first. This paper proposes a new methodology for integrating generative UX research into developing domain NLP applications. Generative UX research employs domain users at the initial stages of prototype development, i.e., ideation and concept evaluation, and the last stage for evaluating system usefulness and user utility. The methodology emerged from and is evaluated on a case study about the full-cycle prototype development of a domain-specific semantic search for daily operations in the process industry. A key finding of our case study is that involving domain experts increases their interest and trust in the final NLP application. The combined UX+NLP research of the proposed method efficiently considers data- and user-driven opportunities and constraints, which can be crucial for developing NLP applications.