ArguAgent: AI-Supported Real-Time Grouping for Productive Argumentation in STEM Classrooms

TL;DR

ArguAgent is an AI system that dynamically groups students in STEM classrooms based on argumentation skills and stances, promoting inclusive and productive discussions.

Contribution

It introduces a real-time AI-powered grouping method that balances stance diversity and argument quality, validated through scoring and simulation experiments.

Findings

ArguAgent's argument scoring aligns well with human experts (Krippendorff's alpha = 0.817).

Prompt engineering significantly improves AI scoring accuracy (QWK from 0.531 to 0.686).

The grouping algorithm achieves 95.4% effectiveness in forming balanced groups.

Abstract

Argumentation is a core practice in STEM education, but its productivity depends on who participates and how they interact. Higher-achieving students often dominate the talk and decision-making, while lower-achieving peers may disengage, defer, or comply without contributing substantive reasoning. Forming groups strategically based on students' stances and argumentation skills could help foster inclusive, evidence-based discourse. In practice, however, teachers are constrained in implementing this grouping strategy because it requires real-time insight into students' positions and the quality of their argumentation, information that is difficult to assess reliably and at scale during instruction. We present a generative AI-powered system, ArguAgent, that creates groups optimizing for stance heterogeneity while constraining argumentation quality differences to +/-1 level on a validated learning progression. ArguAgent uses a two-component assessment pipeline: first scoring student arguments on a 0-4 rubric, then clustering positions via semantic analysis. We validated the scoring component against human expert consensus (Krippendorff's {\alpha}\alpha {\alpha} = 0.817) using 200 expert-generated scores. Testing three OpenAI models (GPT-4o-mini, GPT-5.1, GPT-5.2) with identical calibrated prompts, we found that systematic prompt engineering informed by human disagreement analysis contributed 89% of scoring improvement (QWK: 0.531 to 0.686), while model upgrades contributed an additional 11% (QWK: 0.686 to 0.708). Simulation testing across 100 classes demonstrated that the grouping algorithm achieves 95.4% of groups that meet both design criteria, a 3.2x improvement over random assignment. These results suggest ArguAgent can enable real-time, theoretically grounded grouping that promotes productive STEM argumentation in classrooms.