Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing Development

TL;DR

Prober.ai is a web-based tool that uses constrained LLMs to generate inquiry-based questions for students to reflect on argumentative weaknesses, aiming to enhance critical thinking in writing education.

Contribution

It introduces a novel LLM-constrained, inquiry-based feedback system grounded in argumentation theory, with a two-phase pedagogical interaction architecture.

Findings

Prototype developed in 36 hours at NY EdTech Hackathon

System effectively constrains LLM output to pedagogical JSON schemas

Demonstrates scalable, cognition-preserving AI feedback in writing education

Abstract

The proliferation of large language models (LLMs) in educational settings has paradoxically undermined the cognitive processes they purport to support. Students increasingly outsource critical thinking to AI assistants that generate polished text on demand, resulting in measurable cognitive debt and diminished argumentative reasoning skills. We present Prober.ai, a web-based writing environment that inverts the conventional AI-tutoring paradigm: rather than generating or rewriting student text, the system constrains an LLM (Gemini 3 Flash Preview) through persona-specific system prompts and structured JSON output schemas to produce only targeted, inquiry-based questions about argumentative weaknesses. A two-phase interaction architecture -- Challenge and Unlock -- implements a pedagogical friction mechanism whereby revision suggestions are gated behind mandatory student reflection. The system's design is grounded in Toulmin's argumentation theory, research on peer feedforward questioning mechanisms, and evidence on AI-supported feedback in writing instruction. A functional prototype was developed in 36 hours during the NY EdTech Hackathon (March 2026), where it was awarded second place. We describe the system architecture, the prompt engineering methodology for constraining LLM output to pedagogically aligned JSON schemas, and discuss implications for scalable, cognition-preserving AI integration in writing education.