SocraticAI: Transforming LLMs into Guided CS Tutors Through Scaffolded Interaction

TL;DR

SocraticAI is a scaffolded AI tutoring system that guides undergraduate CS students in responsible, strategic use of LLMs through structured interaction, fostering deeper engagement and problem-solving skills.

Contribution

The paper introduces SocraticAI, a novel scaffolded approach that integrates LLMs into CS education with technical guardrails to promote responsible AI use and strategic thinking.

Findings

Students improved from vague help-seeking to problem decomposition in 2-3 weeks.

Over 75% of students produced substantive reflections.

Students exhibited emergent patterns of deliberate AI use.

Abstract

We present SocraticAI, a scaffolded AI tutoring system that integrates large language models (LLMs) into undergraduate Computer Science education through structured constraints rather than prohibition. The system enforces well-formulated questions, reflective engagement, and daily usage limits while providing Socratic dialogue scaffolds. Unlike traditional AI bans, our approach cultivates responsible and strategic AI interaction skills through technical guardrails, including authentication, query validation, structured feedback, and RAG-based course grounding. Initial deployment demonstrates that students progress from vague help-seeking to sophisticated problem decomposition within 2-3 weeks, with over 75% producing substantive reflections and displaying emergent patterns of deliberate, strategic AI use.