Sensing What Surveys Miss: Understanding and Personalizing Proactive LLM Support by User Modeling

TL;DR

This paper presents a personalized, proactive support system using physiological and behavioral data to improve survey response accuracy and user experience by timely triggering LLM-based assistance.

Contribution

It introduces a novel adaptive system that personalizes support timing based on user state, significantly enhancing survey performance and user perception.

Findings

Aligned-adaptive assistance increased response accuracy by 21%.

False negative help-seeking rates decreased from 50.9% to 22.9%.

Participants reported improved perceived efficiency and trust.

Abstract

Difficulty spillover and suboptimal help-seeking challenge the sequential, knowledge-intensive nature of digital tasks. In online surveys, tough questions can drain mental energy and hurt performance on later questions, while users often fail to recognize when they need assistance or may satisfy, lacking motivation to seek help. We developed a proactive, adaptive system using electrodermal activity and mouse movement to predict when respondents need support. Personalized classifiers with a rule-based threshold adaptation trigger timely LLM-based clarifications and explanations. In a within-subjects study (N=32), aligned-adaptive timing was compared to misaligned-adaptive and random-adaptive controls. Aligned-adaptive assistance improved response accuracy by 21%, reduced false negative rates from 50.9% to 22.9%, and improved perceived efficiency, dependability, and benevolence. Properly timed interventions prevent cascades of degraded responses, showing that aligning support with cognitive states improves both the outcomes and the user experience. This enables more effective, personalized LLM-assisted support in survey-based research.