Behavioral Indicators of Overreliance During Interaction with Conversational Language Models

TL;DR

This study investigates behavioral patterns indicating overreliance on conversational language models, revealing key user behaviors linked to susceptibility to misinformation during interactions.

Contribution

It introduces a novel analysis of user interaction behaviors correlated with overreliance, based on real-world tasks and semantic clustering of interaction segments.

Findings

Low overreliance users demonstrate careful comprehension and navigation.

High overreliance users frequently copy-paste and accept misinformation.

Identified five behavioral patterns associated with overreliance.

Abstract

LLMs are now embedded in a wide range of everyday scenarios. However, their inherent hallucinations risk hiding misinformation in fluent responses, raising concerns about overreliance on AI. Detecting overreliance is challenging, as it often arises in complex, dynamic contexts and cannot be easily captured by post-hoc task outcomes. In this work, we aim to investigate how users' behavioral patterns correlate with overreliance. We collected interaction logs from 77 participants working with an LLM injected plausible misinformation across three real-world tasks and we assessed overreliance by whether participants detected and corrected these errors. By semantically encoding and clustering segments of user interactions, we identified five behavioral patterns linked to overreliance: users with low overreliance show careful task comprehension and fine-grained navigation; users with high overreliance show frequent copy-paste, skipping initial comprehension, repeated LLM references, coarse locating, and accepting misinformation despite hesitation. We discuss design implications for mitigation.