Supporting Business Document Workflows via Collection-Centric Information Foraging with Large Language Models

TL;DR

Marco is a workspace that leverages large language models to assist knowledge workers in efficiently extracting, organizing, and analyzing information from diverse business document collections, reducing cognitive effort.

Contribution

The paper introduces Marco, a novel mixed-initiative system that enhances sensemaking over document collections using collection-centric assistance and natural language interaction.

Findings

Users completed tasks 16% faster with Marco.

Marco reduced user effort without sacrificing accuracy.

Domain experts identified practical workflow benefits.

Abstract

Knowledge workers often need to extract and analyze information from a collection of documents to solve complex information tasks in the workplace, e.g., hiring managers reviewing resumes or analysts assessing risk in contracts. However, foraging for relevant information can become tedious and repetitive over many documents and criteria of interest. We introduce Marco, a mixed-initiative workspace supporting sensemaking over diverse business document collections. Through collection-centric assistance, Marco reduces the cognitive costs of extracting and structuring information, allowing users to prioritize comparative synthesis and decision making processes. Users interactively communicate their information needs to an AI assistant using natural language and compose schemas that provide an overview of a document collection. Findings from a usability study (n=16) demonstrate that when using Marco, users complete sensemaking tasks 16% more quickly, with less effort, and without diminishing accuracy. A design probe with seven domain experts identifies how Marco can benefit various real-world workflows.