RelianceScope: An Analytical Framework for Examining Students' Reliance on Generative AI Chatbots in Problem Solving
Hyoungwook Jin, Minju Yoo, Jieun Han, Zixin Chen, So-Yeon Ahn, Xu Wang

TL;DR
RelianceScope is a new analytical framework that characterizes students' reliance on AI chatbots during problem-solving by analyzing engagement patterns in help-seeking and response-use, accounting for prior knowledge and instructional context.
Contribution
It introduces a systematic method to categorize reliance behaviors in student-AI interactions and demonstrates its application with real student data and LLM detection capabilities.
Findings
Active help-seeking correlates with active response-use.
Reliance patterns are consistent across different knowledge levels.
LLMs can reliably detect reliance behaviors in logs.
Abstract
Generative AI chatbots enable personalized problem-solving, but effective learning requires students to self-regulate both how they seek help and how they use AI-generated responses. Considering engagement modes across these two actions reveals nuanced reliance patterns: for example, a student may actively engage in help-seeking by clearly specifying areas of need, yet engage passively in response-use by copying AI outputs, or vice versa. However, existing research lacks systematic tools for jointly capturing engagement across help-seeking and response-use, limiting the analysis of such reliance behaviors. We introduce RelianceScope, an analytical framework that characterizes students' reliance on chatbots during problem-solving. RelianceScope (1) operationalizes reliance into nine patterns based on combinations of engagement modes in help-seeking and response-use, and (2) situates…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
