"Help Me, But Don't Track Me": Intervention Timing and Privacy Boundaries for Process-Aware AI Tutors

TL;DR

This study explores secondary students' preferences for privacy and support in process-aware AI tutors, highlighting the importance of balancing timely assistance with privacy boundaries and learner autonomy.

Contribution

It provides empirical insights into student preferences for proactive support and privacy boundaries, informing the design of privacy-aware, process-sensitive AI tutoring systems.

Findings

Students prefer hints over direct answers.

Graduated proactive support is favored over constant interruptions.

Students are comfortable sharing problem-solving data but not attention or behavior signals.

Abstract

As generative AI (GenAI) tools are increasingly used as informal tutors for mathematics learning, future systems may become more proactive and process-aware in deciding when and how to offer support. Yet such support raises an important design tension: help that is timely may also feel interruptive or overly monitoring. To inform the design of process-aware AI tutors, we surveyed 330 secondary school students in China (Grades 7--11) about their preferred tutoring behaviors, attitudes toward proactive intervention, and acceptable use of learning-process data. We found three design-relevant patterns. First, students preferred autonomy-preserving support, such as hints over direct answers. Second, they favored graduated proactive support over constant interruption, preferring small hints first and stronger assistance only as needed. Third, they drew clear privacy boundaries around learning-process data: students were comfortable with problem-solving steps and mistake patterns, but substantially less comfortable with attention- or behavior-related signals. Together, these findings offer early empirical guidance for designing AI tutors that balance timely support with learner agency, and personalization with perceived privacy boundaries in K-12 contexts.