Practitioner Voices Summit: How Teachers Evaluate AI Tools through Deliberative Sensemaking

TL;DR

This study explores how K-12 teachers evaluate AI tools through a summit where they developed criteria, revealing mechanisms that support deliberative decision-making and emphasizing the importance of teacher agency.

Contribution

It introduces a framework combining TPACK and agency to understand teachers' evaluative reasoning about AI tools, based on empirical data from a national summit.

Findings

Teachers generated over 200 evaluation criteria.

Five mechanisms support deliberative sensemaking: time, artifacts, collaboration, knowledge-building, safety.

Most teachers view AI as an assistant, not a coach for professional learning.

Abstract

Teachers face growing pressure to integrate AI tools into their classrooms, yet are rarely positioned as agentic decision-makers in this process. Understanding the criteria teachers use to evaluate AI tools, and the conditions that support such reasoning, is essential for responsible AI integration. We address this gap through a two-day national summit in which 61 U.S. K-12 mathematics educators developed personal rubrics for evaluating AI classroom tools. The summit was designed to support deliberative sensemaking, a process we conceptualize by integrating Technological Pedagogical Content Knowledge (TPACK) with deliberative agency. Teachers generated over 200 criteria - initial articulations spanning four higher-order themes (Practical, Equitable, Flexible, and Rigorous) - that addressed both AI outputs and the process of using AI. Criteria contained productive tensions (e.g., personalization versus fairness, adaptability versus efficiency), and the vast majority framed AI as an assistant rather than a coaching tool for professional learning. Analysis of surveys, interviews, and summit discussions revealed five mechanisms supporting deliberative sensemaking: time and space for deliberation, artifact-centered sensemaking, collaborative reflection through diverse viewpoints, knowledge-building, and psychological safety. Across these mechanisms, TPACK and agency operated in a mutually reinforcing cycle - knowledge-building enabled more grounded evaluative judgment, while the act of constructing criteria deepened teachers' understanding of tools. We discuss implications for edtech developers seeking practitioner input, school leaders making adoption decisions, educators and professional learning designers, and researchers working to elicit teachers' evaluative reasoning about rapidly evolving technologies.