What to Make Sense of in the Era of LLM? A Perspective from the Structure and Efforts in Sensemaking

TL;DR

This paper investigates how GPT-4 can assist in complex sensemaking tasks, specifically deciphering fictional terrorist plots, by comparing holistic and step-by-step approaches to enhance human-AI collaboration.

Contribution

It introduces and evaluates two approaches for leveraging GPT-4 in sensemaking tasks, highlighting potential for improved human-AI collaborative workflows.

Findings

GPT-4 can assist in complex sensemaking tasks

Holistic and step-by-step approaches offer different advantages

Preliminary results suggest potential for enhanced collaboration

Abstract

Sensemaking tasks often entail navigating through complex, ambiguous data to construct coherent insights. Prior work has shown that crowds can effectively distribute cognitive load, pooling diverse perspectives to enhance analytical depth. Recent advancements in LLMs have further expanded the toolkit for sensemaking, offering scalable data processing, complex pattern recognition, and the ability to infer and propose meaningful hypotheses. In this study, we explore how LLMs (i.e., GPT-4) can assist in a complex sensemaking task of deciphering fictional terrorist plots. We explore two different approaches for leveraging GPT-4's capabilities: a holistic sensemaking process and a step-by-step approach. Our preliminary investigations open the doors for future research into optimizing human-AI collaborative workflows, aiming to harness the complementary strengths of both for more effective sensemaking in complex scenarios.