Understanding Questions that Arise When Working with Business Documents

TL;DR

This paper investigates the information needs and queries of users working with business documents through user studies, informing the design of AI-assisted document systems that integrate human expertise.

Contribution

It presents empirical insights from user studies on business document assistance needs and explores system design considerations for AI-human collaborative document support.

Findings

Users have diverse, context-specific questions when working with business documents.

Designing effective document assistants requires understanding the roles of human and AI contributions.

Challenges include integrating human expertise seamlessly into AI-driven systems.

Abstract

While digital assistants are increasingly used to help with various productivity tasks, less attention has been paid to employing them in the domain of business documents. To build an agent that can handle users' information needs in this domain, we must first understand the types of assistance that users desire when working on their documents. In this work, we present results from two user studies that characterize the information needs and queries of authors, reviewers, and readers of business documents. In the first study, we used experience sampling to collect users' questions in-situ as they were working with their documents, and in the second, we built a human-in-the-loop document Q&A system which rendered assistance with a variety of users' questions. Our results have implications for the design of document assistants that complement AI with human intelligence including whether particular skillsets or roles within the document are needed from human respondents, as well as the challenges around such systems.