SoulSeek: Exploring the Use of Social Cues in LLM-based Information Seeking

TL;DR

This paper investigates how integrating social cues into LLM-based search systems affects user perceptions and behaviors, highlighting improvements and limitations through a comprehensive design and user study.

Contribution

It introduces SoulSeek, a prototype system that incorporates social cues into LLM search, demonstrating their impact on user experience and revealing design considerations.

Findings

Social cues enhance perceived relevance and trustworthiness.

They promote more reflective information-seeking behaviors.

Current LLM search systems have notable limitations in social understanding.

Abstract

Social cues, which convey others' presence, behaviors, or identities, play a crucial role in human information seeking by helping individuals judge relevance and trustworthiness. However, existing LLM-based search systems primarily rely on semantic features, creating a misalignment with the socialized cognition underlying natural information seeking. To address this gap, we explore how the integration of social cues into LLM-based search influences users' perceptions, experiences, and behaviors. Focusing on social media platforms that are beginning to adopt LLM-based search, we integrate design workshops, the implementation of the prototype system (SoulSeek), a between-subjects study, and mixed-method analyses to examine both outcome- and process-level findings. The workshop informs the prototype's cue-integrated design. The study shows that social cues improve perceived outcomes and experiences, promote reflective information behaviors, and reveal limits of current LLM-based search. We propose design implications emphasizing better social-knowledge understanding, personalized cue settings, and controllable interactions.