Editrail: Understanding AI Usage by Visualizing Student-AI Interaction in Code

TL;DR

Editrail is an interactive tool that helps programming instructors visualize and understand student-AI interactions during coding, enabling better assessment, intervention, and policy formulation.

Contribution

We introduce Editrail, a novel system that visualizes student-AI interactions in code, addressing the lack of visibility instructors face in monitoring AI usage.

Findings

Instructors can accurately detect conflicting AI use patterns.

Editrail enables timely interventions based on student activity.

The system integrates seamlessly into existing coding workflows.

Abstract

Programming instructors have diverse philosophies about integrating generative AI into their classes. Some encourage students to use AI, while others restrict or forbid it. Regardless of their approach, all instructors benefit from understanding how their students actually use AI while writing code. Such insight helps instructors assess whether AI use aligns with their pedagogical goals, enables timely intervention when they find unproductive usage patterns, and establishes effective policies for AI use. However, our survey with programming instructors found that many instructors lack visibility into how students use AI in their code-writing processes. To address this challenge, we introduce Editrail, an interactive system that enables instructors to track students' AI usage, create personalized assessments, and provide timely interventions, all within the workflow of monitoring coding histories. We found that Editrail enables instructors to detect AI use that conflicts with pedagogical goals accurately and to determine when and which students require intervention.